{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 Rental Listing Inquiries 数据上练习 xgboost 参数调优 数据说明： Rental Listing Inquiries 数据集是 Kaggle 平台上的一个分类竞赛任务，需要根据公寓的特征来预测其受欢迎程度（用户感兴趣程度分为高、中、低三类）。其中房屋的特征 x 共有 14 维，响应值 y 为用户对该公寓的感兴趣程度。评价标准为 logloss。 问题描述 采用xgboost模型完成商品分类（需进行参数调优）。 解题提示 为减轻大家对特征工程的入手难度，以及统一标准，数据请用课程网站提供的特征工程编码后的数据（RentListingInquries_FE_train.csv）或稀疏编码的形式（RentListingInquries_FE_train.bin）。xgboost既可以单独调用，也可以在sklearn框架下调用。大家可以随意选择。若采用xgboost单独调用使用方式，建议读取稀疏格式文件。 批改标准 独立调用xgboost或在sklearn框架下调用均可。\n",
    "模型训练：超参数调优 a) 初步确定弱学习器数目： 20分 b) 对树的最大深度（可选）和min_children_weight进行调优（可选）：20分 c) 对正则参数进行调优：20分 d) 重新调整弱学习器数目：10分 e) 行列重采样参数调整：10分\n",
    "调用模型进行测试10分\n",
    "生成测试结果文件10分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 调整树的参数：subsample 和 colsample_bytree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#读取数据\n",
    "train = pd.read_csv(\"RentListingInquries_FE_train.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# drop ids and get labels\n",
    "y_train = train['interest_level']\n",
    "train = train.drop([\"interest_level\"], axis=1)\n",
    "X_train = np.array(train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二轮参数调整得到的n_estimators最优值（250），其余参数继续默认值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'subsample': [0.3, 0.4, 0.5, 0.6, 0.7, 0.8],\n",
       " 'colsample_bytree': [0.6, 0.7, 0.8, 0.9]}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#max_depth 建议3-10， min_child_weight=1／sqrt(ratio_rare_event) =5.5\n",
    "subsample = [i/10.0 for i in range(3,9)]\n",
    "colsample_bytree = [i/10.0 for i in range(6,10)]\n",
    "param_test3_1 = dict(subsample=subsample, colsample_bytree=colsample_bytree)\n",
    "param_test3_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.58904, std: 0.00375, params: {'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  mean: -0.58682, std: 0.00427, params: {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  mean: -0.58546, std: 0.00324, params: {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  mean: -0.58460, std: 0.00323, params: {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  mean: -0.58417, std: 0.00338, params: {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  mean: -0.58324, std: 0.00292, params: {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  mean: -0.58800, std: 0.00376, params: {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  mean: -0.58649, std: 0.00420, params: {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  mean: -0.58486, std: 0.00409, params: {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  mean: -0.58457, std: 0.00421, params: {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  mean: -0.58362, std: 0.00417, params: {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  mean: -0.58359, std: 0.00419, params: {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  mean: -0.58829, std: 0.00436, params: {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  mean: -0.58693, std: 0.00470, params: {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  mean: -0.58556, std: 0.00415, params: {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  mean: -0.58451, std: 0.00357, params: {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  mean: -0.58384, std: 0.00365, params: {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  mean: -0.58320, std: 0.00335, params: {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  mean: -0.58708, std: 0.00443, params: {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  mean: -0.58624, std: 0.00407, params: {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  mean: -0.58464, std: 0.00338, params: {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  mean: -0.58377, std: 0.00391, params: {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  mean: -0.58282, std: 0.00372, params: {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  mean: -0.58266, std: 0.00396, params: {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " {'colsample_bytree': 0.9, 'subsample': 0.8},\n",
       " -0.58266324519599)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb3_1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=250, \n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "gsearch3_1 = GridSearchCV(xgb3_1, param_grid = param_test3_1, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch3_1.fit(X_train , y_train)\n",
    "\n",
    "gsearch3_1.grid_scores_, gsearch3_1.best_params_,     gsearch3_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([705.09464693, 737.5659533 , 765.21025343, 778.99027014,\n",
       "        786.96042829, 778.38172894, 745.41114259, 801.43511496,\n",
       "        835.03939681, 856.71592035, 853.46385837, 841.3362309 ,\n",
       "        797.15546513, 832.99212861, 873.29258933, 893.59641085,\n",
       "        906.15487103, 895.56636   , 866.10040374, 927.03680439,\n",
       "        923.51389122, 795.53965483, 781.59762373, 772.33806062]),\n",
       " 'std_fit_time': array([ 5.16065769,  7.05783508,  4.39974964,  2.21861091,  2.44974595,\n",
       "         7.12036862,  3.73812522,  4.20325923,  8.10742541,  3.32721665,\n",
       "         2.73278875,  5.78052647,  8.89880219,  7.63235984,  5.63933405,\n",
       "         2.37767271,  0.83903215, 12.31010183,  5.41507529,  5.88431779,\n",
       "        51.72307851, 15.56525426,  5.22225051, 24.48246458]),\n",
       " 'mean_score_time': array([1.73676848, 1.35800047, 1.36761694, 1.40437331, 1.40895739,\n",
       "        1.43627396, 1.44988103, 1.44257665, 1.5530118 , 1.48296056,\n",
       "        1.47375579, 1.40036998, 1.43482542, 1.37786498, 1.42803359,\n",
       "        1.37219019, 1.54583654, 1.42467661, 1.6050797 , 1.50369458,\n",
       "        1.371029  , 1.03999934, 1.09346828, 1.27600856]),\n",
       " 'std_score_time': array([0.39609556, 0.08611641, 0.04998983, 0.06528738, 0.11983261,\n",
       "        0.04826172, 0.10238388, 0.07152151, 0.23413503, 0.11396413,\n",
       "        0.07285216, 0.0646774 , 0.00864189, 0.05044588, 0.08569725,\n",
       "        0.04988238, 0.07546551, 0.08890134, 0.06234515, 0.08850216,\n",
       "        0.31325151, 0.04242593, 0.10015927, 0.37538581]),\n",
       " 'param_colsample_bytree': masked_array(data=[0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.7, 0.7, 0.7, 0.7, 0.7,\n",
       "                    0.7, 0.8, 0.8, 0.8, 0.8, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9,\n",
       "                    0.9, 0.9],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_subsample': masked_array(data=[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.3, 0.4, 0.5, 0.6, 0.7,\n",
       "                    0.8, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.3, 0.4, 0.5, 0.6,\n",
       "                    0.7, 0.8],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " 'split0_test_score': array([-0.58365176, -0.58021648, -0.58080049, -0.5791575 , -0.57801683,\n",
       "        -0.57841047, -0.58203742, -0.5794007 , -0.57835109, -0.57774432,\n",
       "        -0.57653301, -0.57599825, -0.58050051, -0.57927467, -0.57851961,\n",
       "        -0.57866592, -0.57850172, -0.57740155, -0.57927676, -0.57940048,\n",
       "        -0.57880306, -0.57729755, -0.57733812, -0.57689102]),\n",
       " 'split1_test_score': array([-0.5867409 , -0.58446499, -0.58283012, -0.58319513, -0.58321485,\n",
       "        -0.58162064, -0.5855263 , -0.58441483, -0.58332037, -0.58286071,\n",
       "        -0.58196033, -0.58270294, -0.58677278, -0.58499787, -0.58398358,\n",
       "        -0.58265377, -0.58135493, -0.58199138, -0.58568248, -0.58404288,\n",
       "        -0.58337935, -0.58332875, -0.58079429, -0.58080558]),\n",
       " 'split2_test_score': array([-0.58896107, -0.58680239, -0.58593168, -0.58514132, -0.58555004,\n",
       "        -0.58399958, -0.58924634, -0.58788258, -0.58429078, -0.58481165,\n",
       "        -0.58428569, -0.58460916, -0.59014612, -0.58742526, -0.58626312,\n",
       "        -0.58532429, -0.5836675 , -0.58393661, -0.58804644, -0.58785728,\n",
       "        -0.58532389, -0.58261137, -0.58207698, -0.58136078]),\n",
       " 'split3_test_score': array([-0.59119881, -0.59224851, -0.58844778, -0.58825426, -0.58698725,\n",
       "        -0.58616257, -0.59059818, -0.58958474, -0.58836165, -0.58721876,\n",
       "        -0.58711808, -0.58678667, -0.59122814, -0.58968218, -0.58851596,\n",
       "        -0.58701837, -0.58767271, -0.58569368, -0.59144482, -0.59063053,\n",
       "        -0.58743411, -0.5883092 , -0.5873223 , -0.58711281]),\n",
       " 'split4_test_score': array([-0.59463347, -0.59038453, -0.58927689, -0.58725754, -0.58706257,\n",
       "        -0.58601288, -0.59257149, -0.59117529, -0.58997626, -0.59023044,\n",
       "        -0.58822278, -0.58783048, -0.59278401, -0.59328421, -0.5905319 ,\n",
       "        -0.58887568, -0.58800018, -0.58697801, -0.59097569, -0.58926569,\n",
       "        -0.58825197, -0.58729528, -0.58656893, -0.5871474 ]),\n",
       " 'mean_test_score': array([-0.58903686, -0.58682316, -0.58545716, -0.58460099, -0.58416613,\n",
       "        -0.58324106, -0.58799567, -0.58649134, -0.58485972, -0.58457283,\n",
       "        -0.5836237 , -0.58358524, -0.58828604, -0.58693245, -0.58556253,\n",
       "        -0.58450734, -0.58383916, -0.58320002, -0.587085  , -0.58623919,\n",
       "        -0.58463826, -0.58376821, -0.5828199 , -0.58266325]),\n",
       " 'std_test_score': array([0.00374879, 0.00427386, 0.00323555, 0.00323434, 0.00337589,\n",
       "        0.00292418, 0.00376489, 0.004196  , 0.00408644, 0.00421004,\n",
       "        0.00416759, 0.00418713, 0.00436467, 0.00470165, 0.00414733,\n",
       "        0.00356682, 0.00365234, 0.00335011, 0.00442734, 0.00406585,\n",
       "        0.00337579, 0.00391056, 0.00371569, 0.00395906]),\n",
       " 'rank_test_score': array([24, 19, 15, 12,  9,  4, 22, 18, 14, 11,  6,  5, 23, 20, 16, 10,  8,\n",
       "         3, 21, 17, 13,  7,  2,  1]),\n",
       " 'split0_train_score': array([-0.50958664, -0.50765189, -0.50551885, -0.50480425, -0.50306946,\n",
       "        -0.50480578, -0.50642375, -0.5036021 , -0.50211168, -0.50166327,\n",
       "        -0.50202364, -0.50165641, -0.50494877, -0.50226311, -0.49939752,\n",
       "        -0.49930209, -0.49968822, -0.49984747, -0.50297773, -0.49931704,\n",
       "        -0.49778332, -0.4952909 , -0.49571846, -0.49728608]),\n",
       " 'split1_train_score': array([-0.50985928, -0.50625499, -0.50498779, -0.50428869, -0.50343587,\n",
       "        -0.50286724, -0.50631423, -0.50351889, -0.50227296, -0.49961694,\n",
       "        -0.5007591 , -0.50115912, -0.50431737, -0.50076077, -0.49816103,\n",
       "        -0.49713861, -0.49825608, -0.4991744 , -0.50209665, -0.49842074,\n",
       "        -0.49879955, -0.49746083, -0.49479965, -0.49826303]),\n",
       " 'split2_train_score': array([-0.50859995, -0.50532666, -0.503349  , -0.50240285, -0.50201243,\n",
       "        -0.50405562, -0.50576737, -0.50319951, -0.50093325, -0.50088373,\n",
       "        -0.50079099, -0.50168477, -0.50353721, -0.49986244, -0.49819222,\n",
       "        -0.49754117, -0.49809137, -0.49785481, -0.50012691, -0.49828878,\n",
       "        -0.49566945, -0.49612973, -0.4966592 , -0.49660617]),\n",
       " 'split3_train_score': array([-0.50838763, -0.50755423, -0.50325582, -0.50152474, -0.50249862,\n",
       "        -0.50357847, -0.50631473, -0.50148685, -0.50055835, -0.50109922,\n",
       "        -0.50122166, -0.5008994 , -0.50421757, -0.50005538, -0.49849709,\n",
       "        -0.49800814, -0.49779447, -0.4975287 , -0.50089559, -0.49708742,\n",
       "        -0.49537031, -0.49385938, -0.49666628, -0.49705754]),\n",
       " 'split4_train_score': array([-0.50891086, -0.50503588, -0.50471936, -0.5030843 , -0.50171496,\n",
       "        -0.50262266, -0.50410067, -0.49957805, -0.49999715, -0.50086398,\n",
       "        -0.5014696 , -0.50130997, -0.50203397, -0.49884629, -0.4970648 ,\n",
       "        -0.49678731, -0.49617741, -0.49904174, -0.50015092, -0.49832749,\n",
       "        -0.49680864, -0.49434158, -0.49505574, -0.49683841]),\n",
       " 'mean_train_score': array([-0.50906887, -0.50636473, -0.50436616, -0.50322096, -0.50254627,\n",
       "        -0.50358595, -0.50578415, -0.50227708, -0.50117468, -0.50082543,\n",
       "        -0.501253  , -0.50134193, -0.50381098, -0.5003576 , -0.49826253,\n",
       "        -0.49775546, -0.49800151, -0.49868942, -0.50124956, -0.49828829,\n",
       "        -0.49688625, -0.49541648, -0.49577987, -0.49721025]),\n",
       " 'std_train_score': array([0.000566  , 0.00108877, 0.00090635, 0.001201  , 0.00063929,\n",
       "        0.00079412, 0.00087247, 0.00155293, 0.00088418, 0.00066967,\n",
       "        0.00046887, 0.0002989 , 0.00099489, 0.00113261, 0.0007474 ,\n",
       "        0.00087403, 0.00112144, 0.00086536, 0.00112221, 0.00070986,\n",
       "        0.00128453, 0.00128715, 0.00078076, 0.00057282])}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch3_1.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.582663 using {'colsample_bytree': 0.9, 'subsample': 0.8}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XdYVMfXwPHv0EREmlhQVLAXmr1ijS22iIqxoFijiUlMMWL8WVM0iUmMUWOsGLugxhZ772JBLBhFIYoiImBDkLLz/rHICwJSVxHn8zz7ZMvMvXM3wuHOvXOOkFKiKIqiKLml97oHoCiKorzZVCBRFEVR8kQFEkVRFCVPVCBRFEVR8kQFEkVRFCVPVCBRFEVR8kQFEkVRFCVPVCBRFEVR8kQFEkVRFCVPDF73AF4Fa2traWdn97qHoSiK8kY5c+bMfSllyazavRWBxM7OjtOnT7/uYSiKorxRhBD/ZaedmtpSFEVR8kQFEkVRFCVPVCBRFEVR8uStuEaiKAVBQkICoaGhxMXFve6hKEoaxsbG2NraYmhomKv+KpAoyisSGhpK8eLFsbOzQwjxuoejKABIKYmMjCQ0NBR7e/tcbUNNbSnKKxIXF0eJEiVUEFEKFCEEJUqUyNOZsk4DiRCioxDiXyFEkBDCK4PPPYUQEUII/+THsFSf/SiEuCSECBRCzBbJP31CiD5CiIDkz37U5fgVJb+pIKIURHn9d6mzQCKE0AfmAp2AWkBfIUStDJqulVK6JD8WJfdtCjQDnAAHoAHQUghRAvgJaCulrA2UFkK01dUxrD51kwP/3tPV5hVFUQoFXZ6RNASCpJQ3pJTxwBqgezb7SsAYMAKKAIZAOFAJuCqljEhutwfoma+jThafqGHFif/4cOVZLoQ+1MUuFEVRCgVdBpJywK1Ur0OT33tRz+SpKl8hRHkAKeVxYD8QlvzYKaUMBIKAGkIIOyGEAfAeUD6jnQshRgghTgshTkdERGTU5KWMDPRY6tkASxMjBnv7cSvqaY63oSgFyYMHD5g3b16u+s6aNYunT/PvZ8Db25vRo0fn2/ay68CBA3Tp0iXH/XI63pCQEFatWpXj/WRXVFQU7dq1o2rVqrRr147o6OgM2928eZP27dtTs2ZNatWqRUhIiE7Go8tAktGkm3zh9RbATkrphPbsYhmAEKIKUBOwRRt82gghWkgpo4FRwFrgMBACJGa0cynlAillfSll/ZIls0wVk6FSZsYsG9KAhCQNg5acIiomPlfbUZSCoCAFksLuZYEkMTHDX1k5MmPGDNq2bcu1a9do27YtM2bMyLDdwIEDGTt2LIGBgZw6dYpSpUrled8Z0eXtv6GkPVuwBe6kbiCljEz1ciHwQ/LzHsAJKeUTACHEdqAxcEhKuQVtAEIIMQJI0snok1UpVZzFg+rTb9FJhi3zY+WwxhQ10tflLpW3wNQtl7h851G+brNWWTMmd62d6edeXl5cv34dFxcX2rVrR6lSpVi3bh3Pnj2jR48eTJ06lZiYGNzd3QkNDSUpKYmJEycSHh7OnTt3aN26NdbW1uzfvz/D7e/YsYOvv/6apKQkrK2t2bt3L1FRUQwZMoQbN25gYmLCggULcHJyStPPx8eHqVOnoq+vj7m5OYcOHSIkJAQPDw9iYmIAmDNnDk2bNuXAgQNMnjyZ0qVL4+/vj5ubG46Ojvz222/Exsby999/U7lyZTw9PTE2NubSpUuEh4fzyy+/pDsTiYmJ4eOPP+bChQskJiYyZcoUunfPfPb91q1bdOzYkeDgYPr168fkyZOZOHEi1tbWfPrppwBMmDCB0qVLs2rVKgIDA3FxcWHQoEFYWlqybds24uLiiImJYd++ffz000/pvn+AFStWMHv2bOLj42nUqBHz5s1DXz/t75xNmzZx4MABAAYNGkSrVq344Ycf0rS5fPkyiYmJtGvXDgBTU9NMjy2vdBlI/ICqQgh74DbwPtAvdQMhhI2UMiz5ZTcgMPn5TWC4EGI62jOblsCs5D6lpJT3hBCWwIeAuw6PAYD6dlbMft+FUSvP8umac/wxoB76euruG+XNMmPGDC5evIi/vz+7du3C19eXU6dOIaWkW7duHDp0iIiICMqWLcu2bdsAePjwIebm5vzyyy/s378fa2vrDLcdERHB8OHDOXToEPb29kRFRQEwefJk6tSpw99//82+ffsYOHAg/v7+afpOmzaNnTt3Uq5cOR48eABAqVKl2L17N8bGxly7do2+ffumJF49f/48gYGBWFlZUalSJYYNG8apU6f47bff+P3335k1axagPSs4ePAg169fp3Xr1gQFBaXZ73fffUebNm1YsmQJDx48oGHDhrzzzjsUK1Ysw2M8deoUFy9exMTEhAYNGtC5c2eGDh2Km5sbn376KRqNhjVr1nDq1CmcnJyYOXMmW7duBbRTY8ePHycgIAArKyt27drFtWvX0n3/JUuWZO3atRw9ehRDQ0M+/PBDVq5cycCBAxk2bBgjR46kfv36hIeHY2NjA4CNjQ337qW/Kejq1atYWFjg5uZGcHAw77zzDjNmzEgXlPKDzgKJlDJRCDEa2AnoA0uklJeEENOA01LKzcAnQohuaKenogDP5O6+QBvgAtrpsB3JZyIAvwkhnJOfT5NSXtXVMaTW0cGGyV1qMWXLZaZsvsS07rXVrZxKrr3szOFV2LVrF7t27aJOnToAPHnyhGvXruHq6sqXX37JuHHj6NKlC66urtna3okTJ2jRokXKgjYrKysAjhw5wvr16wFo06YNkZGRPHyY9uaVZs2a4enpibu7O25uboA2C8Do0aPx9/dHX1+fq1f//8e8QYMGKb9EK1euTPv27QFwdHRMc7bk7u6Onp4eVatWpVKlSly5ciXdd7B582ZmzpwJaNf53Lx5k5o1a2Z4jO3ataNEiRIAuLm5ceTIEcaMGUOJEiU4d+4c4eHh1KlTJ6VNRv2ffy+Zff8BAQGcOXOGBg0aABAbG5syHbVo0aIMt5uZxMREDh8+zLlz56hQoQJ9+vTB29uboUOH5mg72aHTle1Syn+Af154b1Kq5+OB8Rn0SwI+yGSbffN5mNnm2cyesIdx/HnoBjYWxnzYqsrrGoqi5ImUkvHjx/PBB+l/zM6cOcM///zD+PHjad++PZMmTcpgC+m3l9EfVlK+eFk0/ZqF+fPnc/LkSbZt24aLiwv+/v78/vvvlC5dmvPnz6PRaDA2Nk5pX6RIkZTnenp6Ka/19PTSXH94cT8vvpZSsn79eqpXr57l8b1se8OGDcPb25u7d+8yZMiQTPunPtPJ7Pv//fffGTRoENOnT3/pWEqXLk1YWBg2NjaEhYVleO3D1taWOnXqUKlSJQDee+89Tpw4oZNAola259C4jjXo5lyWH3f8y4azoa97OIqSbcWLF+fx48cAdOjQgSVLlvDkyRMAbt++zb1797hz5w4mJiYMGDCAL7/8krNnz6brm5EmTZpw8OBBgoODAVKmtlq0aMHKlSsB7R1T1tbWmJmZpel7/fp1GjVqxLRp07C2tubWrVs8fPgQGxsb9PT0WL58OUlJOb8U6uPjg0aj4fr169y4cSNdwOjQoQO///57SrA7d+7cS7e3e/duoqKiUq7FNGvWDIAePXqwY8cO/Pz86NChA5D195XZ99+2bVt8fX1TpqqioqL477/0JUG6devGsmXLAFi2bFmG13YaNGhAdHQ0z+9a3bdvH7VqZbSUL+9Urq0c0tMT/NTbiYjHz/jKN4BSxY1pXjXjeWNFKUhKlChBs2bNcHBwoFOnTvTr148mTZoA2guxK1asICgoiLFjx6Knp4ehoSF//PEHACNGjKBTp07Y2NhkeLG9ZMmSLFiwADc3NzQaTco1jilTpjB48GCcnJwwMTFJ+eWX2tixY7l27RpSStq2bYuzszMffvghPXv2xMfHh9atW2d63eJlqlevTsuWLQkPD2f+/PlpzmoAJk6cyJgxY3ByckJKiZ2dXco1jYw0b94cDw8PgoKC6NevH/Xr1wfAyMiI1q1bY2FhkXL9wcnJCQMDA5ydnfH09MTS0jLNttq3b09gYGC6779WrVp8++23tG/fHo1Gg6GhIXPnzqVixYpprpF4eXnh7u7O4sWLqVChAj4+PgCcPn2a+fPns2jRIvT19Zk5cyZt27ZFSkm9evUYPnx4jr/H7BAZnXoWNvXr15f5XSHxYWwC7vOPc/tBLGs/aEztsub5un2l8AkMDMx0/l3JX56ennTp0oVevXrpfF8ajYa6devi4+ND1apVdb4/Xcno36cQ4oyUsn5WfdXUVi6ZFzXEe0gDihsbMHipH6HR6h57RXnbXL58mSpVqtC2bds3OojklTojyaN/7z6m1/xjlDYzxndkEyxMjHSyH+XNV1jOSBo1asSzZ8/SvLd8+XIcHR1f04jyz86dOxk3blya9+zt7dm4ceNrGtGrk5czEhVI8sHx65EMWnIKl/IW/DW0IcaGasGikl5hCSRK4aSmtl6zJpVLMNPdmVMhUXy+zh+NpvAHZ0VRlOdUIMkn3ZzLMuHdmvxz4S7fbgvMuoOiKEohoW7/zUfDXO258zCWJUeDKWthzDDXSq97SIqiKDqnAkk+EkIwsXMtwh/F8e22QEqbGdPVuezrHpaiKIpOqamtfKanJ/jF3YWGdlZ8se48x69HZt1JUV6BgpRGXtUjyZvs1CPZv38/Li4uKQ9jY2P+/vtvnYxHBRIdMDbUZ8HAelQoYcKI5af5927mqRIU5VUpSIGksCsI9Uhat26Nv78//v7+7Nu3DxMTk5QEl/lNTW3piIWJEcuGNKTH3KN4Lj3Fhg+bYmNe9HUPSykotnvB3Qv5u80yjtAp4wJHoOqRvG31SFLz9fWlU6dOmJiYZNomL1Qg0aFyFkXxHtwQ9z+PM3ipH+tGNsHM2PB1D0t5S6l6JG9XPZLU1qxZw+eff/7SNnmhAomO1SprxvwB9fBceooP/jqD95AGFDFQCxbfei85c3gVVD2Swl+P5LmwsDAuXLiQkplYF1QgeYmL9y9SzrQclsaWWTd+ieZVrfmptxOfrT3PWJ8AZvVxQU9VWFReI1WPpPDXI3lu3bp19OjRA0ND3c2GqIvtmUjQJPDFgS9w2+zGodBDed5ejzq2fNWxOpvP3+GHHVey7qAo+UzVI3m76pE8t3r1avr21W09QHVGkglDPUNmt5nN+CPj+WjvR/Ss2pOvGnyFiWHuL1aNalmZsAfJFRbNjfFsZp+PI1aUl1P1SN6ueiSgvU5069YtWrZsmePvLydU0sYsxCfFM9d/LksvLqWcaTm+a/4ddUvXzfVYkjSSUSvOsDswnHn96tLJ0SbX21LeLCpp46uj6pHknEraqENG+kZ8Vu8zvDt6I5F47vDk1zO/Ep8Un6vt6esJZvetQ53yFny61h+/kKh8HrGiKK+Kqkeipc5IciAmIYaf/H5i/bX1VLOsxvfNv6e6VfYu1L0oOiaenn8cIzImnvWjmlClVPE8j08p2ArLGYmqR1I4qXokWcjveiQHbx1k8rHJPIx/yGiX0XjW9kRfL+e39N6KekqPeccoYqDHxg+bUsrMOOtOyhursAQSpXAqsFNbQoiOQoh/hRBBQgivDD73FEJECCH8kx/DUn32oxDikhAiUAgxWyTfayeE6CuEuCCECBBC7BBCZLxCSodalm/Jxu4baV2+NbPOzmLwzsHcenwrx9spb2XCUs8GRD+Nx3OpH4/jEnQwWkVRFN3SWSARQugDc4FOQC2grxCiVgZN10opXZIfi5L7NgWaAU6AA9AAaCmEMAB+A1pLKZ2AAODVZ34DLI0t+bnlz3zf/HuCooPoubknvld9M7xv/mUcbc2Z178u/4Y/5sOVZ4lP1OhoxIqiKLqhyzOShkCQlPKGlDIeWANkfrNzWhIwBoyAIoAhEA6I5Eex5DMUM+BOfg88xZ1zEJN59l4hBF0rd2VD9w04WTsx9fhURu8bzf3Y+znaTavqpZjh5sjha/fx2hCQ42CkKIryOukykJQDUs/3hCa/96KeydNUvkKI8gBSyuPAfiAs+bFTShkopUwARgEX0AaQWsDijHYuhBghhDgthDgdERGR89EnJcDagfCnK9zye2nTMsXKsKD9ArwaenEy7CQ9NvVg93+7c7S73vXL83m7amw4e5uZu/7N+XgVJQsFKfuvSiOfN9lJIw/w1VdfUbt2bWrWrMknn3yisz9SdRlIMsoB8uJRbAHskqep9gDLAIQQVYCagC3a4NNGCNFCCGGINpDUAcqindoan9HOpZQLpJT1pZT1S5YsmfPR6xtCn+WgZwBLO8KJ+fCS/wl6Qo/+Nfuzrus6ypmW4/MDnzP+8HgexT/K9i4/blOFvg3LM3f/dVacSL+aVVHyoiAFksKuIKSRP3bsGEePHiUgIICLFy/i5+fHwYMH87zvjOgykIQC5VO9tuWFaSgpZaSU8vl9hAuBesnPewAnpJRPpJRPgO1AY8Alud91qQ2t64CmOjuCsi7wwUGo2h52jAMfT4h7eWCoZF6J5e8u50PnD9kevB23TW6cCDuRrd0JIfimuwNta5Ri0qaL7Lp0Nx8OQlG0UqeRHzt2LD/99BMNGjTAycmJyZMnA9rU6p07d8bZ2RkHBwfWrl3L7NmzU9LIt27dOtPt79ixg7p16+Ls7Ezbtm0B7V/O7733Hk5OTjRu3JiAgIB0/Xx8fHBwcMDZ2ZkWLVoA2l/Erq6u1K1bl7p163Ls2DFAe0bRsmVL3N3dqVatGl5eXqxcuZKGDRvi6OjI9evXAe2CxJEjR+Lq6kq1atUyXLEeExPDkCFDaNCgAXXq1GHTpk0v/f6ep5GvXr16Ssr3iRMn8ttvv6W0mTBhArNnz8bLy4vDhw/j4uLCr7/+ire3N71796Zr164pSSYz+v5Bm0a+YcOGuLi48MEHH2SYHmbTpk0MGjQI0KaRz6hglRCCuLg44uPjefbsGQkJCZQuXfqlx5hbukyR4gdUFULYA7eB94F+qRsIIWyklGHJL7sBgcnPbwLDhRDT0Z7ZtARmJW+nlhCipJQyAmiXqo9uFLWE91fB0d9g7zQIvwjuf0Hp2pl2MdQzZJTLKFxtXRl/eDzDdw2nf83+jKk7BmODl9/ia6Cvx+/96tB34Uk+WXOOVcMbU7dC3pJGKgXPD6d+4EpU/uZcq2FVg3ENx2X6uUoj/3alkW/SpAmtW7fGxsYGKSWjR4/W2e3nOjsjkVImor2jaifaX/brpJSXhBDThBDdkpt9knyL73ngE8Az+X1f4DraayHngfNSyi1SyjvAVOCQECIA7RnK97o6hhRCQPMxMGgzPHsMC9vC+TVZdnOwdmBd13X0r9mflYErcd/qzsX7F7PsZ2JkwOJB9SltZsxQbz9uRDzJj6NQlBSp05jXrVuXK1eucO3aNRwdHdmzZw/jxo3j8OHDmJubZ2t7L0sj7+HhAWSdRn7hwoUpf30nJCQwfPhwHB0d6d27N5cvX05p/zyNfJEiRdKlkQ8JCUlpl5008jNmzMDFxYVWrVqlpJHPzPM08kWLFk1JI29nZ5eSRv7595nTNPKpv/+9e/empJF3cXFh79693LhxA9CmkX+e3ys7goKCCAwMJDQ0lNu3b7Nv3z4OHcp7AtqM6DRpo5TyH+CfF96blOr5eDK4xiGlTALS57fWfjYfmJ+/I80mu+bwwWFYPxQ2fgA3j0PHH8Aw87OMogZF8WroRUvblkw8OpEB/wxghNMIhjsNx1Av87TO1qZFWDa4IT3/OMagpafYMKoZJYsXybS98mZ52ZnDq6DSyBf+NPIbN26kcePGmJqaAtCpU6eUgJ/fVK6tnCpeGjz+huafwRlvWNwOooKz7NakbBM2dN9AJ/tO/HH+Dzz+8eDGwxsv7WNnXYzFng24/zieId5+xDzL+0U65e2l0si/XWnkK1SowMGDB0lMTCQhIYGDBw++eVNbhZq+AbwzBfqugQf/wYKWcOWfrHphZmTGdNfp/NzyZ24/uY37FndWBq5EIzNfhOhS3oI5/epw6c5DPlp1loQktWBRyZ3UaeR3796dkkbe0dGRXr168fjxYy5cuJByofe7777jf//7H/D/aeQzu9ieOo28s7Mzffr0AWDKlCmcPn0aJycnvLy8Mk0j7+joiIODAy1atEhJI79s2TIaN27M1atX85RGvlOnTpmmkU9ISMDJyQkHBwcmTpz40u09TyPv4uJCz54906WRd3d3zzCN/K+//ppuW+3bt8/w+0+dRt7JyYl27doRFqa9jDxs2LCU60ReXl7s3r2bqlWrsnv3bry8tIlDTp8+zbBh2gQhvXr1onLlyjg6OuLs7IyzszNdu3bN8feYHSrXVl5Fh8C6gRB2HpqNgTYTtYEmCxFPI5h8bDKHbx+mkU0jvm32LWWKlcm0/epTNxm/4QLu9W35oadThtMISsGmcm29OiqNfM4V2FxbbwVLOxiyC+oPgaOz4K9u8Djr23ZLmpRkbtu5TG4ymYCIANw2ubHl+pZMFwz1bViBT9pUYd3pUGbtuZbPB6EoSm6oNPJa6owkP51fA1s/AyNT6LUE7F2z1e3W41tMODKBc/fO0a5iOyY2nphhnXgpJV/5BuBzJpQZbo6837BCfh+BokOF5YxEpZEvnFQa+Sy8skACEH5ZO9UVdR3aToKmn4Je1id+SZokvC95M8d/DuZG5kxrNo0WtunvrkhI0jBs2WmOBN1n0cD6tK6R/m4NpWAqLIFEKZzU1FZBUroWjNgPtbrDnimwph/EZpwHJzV9PX2GOg5lTec1WBW14qO9HzHl2BSeJqRNS2Gor8e8/nWpaVOcD1ee5fytBzo6EEVRlOxRgUQXihSHXkuh008QtAf+bKHNJJwN1a2qs6bzGoY4DGHDtQ303NyTs+Fn07QpVsSAJZ4NKGFqxBBvP/6LjNHFUSiKomSLCiS6IgQ0GgGDt4NGA4vbw+klL038+FzqOvFAhnXiSxU3ZtmQhmikZNCSU0Q+eZbJ1hRFUXRLBRJdK98APjgEdq7aC/EbR0J89s4g6paui283X9yqurHk4hL6buvLv1H/n2K+cklTFg1qQNjDOIYuO01sfM4XbSmKouSVCiSvQrES0N8XWk+AgLXaXF33s3cLbzHDYkxpOoW5becSGRvJ+9veZ/GFxSRptEGjXkVLZvetQ0DoAz5efZZEtWBRyURBSiOv6pHkTXbrkYwbNw4HB4eUTM66ogLJq6KnBy2/Ao8NEHMPFrSCixuy3b2FbYtM68R3qF2Gqd1qsyfwHpM2X1IVFpUMFaRAUtgVhHok27Zt4+zZs/j7+3Py5El++uknHj3Kfn2knFCB5FWr3Eab+LFULfAdDNvHQWJ81v14eZ14jyZ2jGpVmVUnbzJ3f1DWG1PeOqoeSVqFvR7J5cuXadmyJQYGBhQrVgxnZ2d27Njx0mPMNSlloX/Uq1dPFjiJ8VJuHy/lZDMpF7SRMvpmjrqHPQmTQ3cOlQ7eDvLDPR/KiKcRMilJIz9dfVZWHLdV+py+paOBK7l1+fLllOdh330nQwZ45Osj7LvvXrr/4OBgWbt2bSmllDt37pTDhw+XGo1GJiUlyc6dO8uDBw9KX19fOWzYsJQ+Dx48kFJKWbFiRRkREZHptu/duydtbW3ljRs3pJRSRkZGSimlHD16tJwyZYqUUsq9e/dKZ2dnKaWUS5culR999JGUUkoHBwcZGhoqpZQyOjpaSillTEyMjI2NlVJKefXqVfn8Z3j//v3S3Nxc3rlzR8bFxcmyZcvKSZMmSSmlnDVrlvz000+llFIOGjRIdujQQSYlJcmrV6/KcuXKydjYWLl//37ZuXNnKaWU48ePl8uXL0/Zb9WqVeWTJ08yPL6lS5fKMmXKyPv378unT5/K2rVrSz8/PxkcHCzr1KkjpZQyKSlJVqpUSd6/fz/Nfp73L1euXMr3ktn3f/nyZdmlSxcZHx8vpZRy1KhRctmyZVJKKYcOHSr9/PyklFKam5unGZ+FhUW6Me/cuVM2bdpUxsTEyIiICGlvby9nzpyZ2f/CNP8+nwNOy2z8jtVpGnnlJfQNoeP3UKER/P2R9hbhnguhyjvZ6l6mWBkWtFvA6iur+fXMr/TY1IOJjSfyY693iHjyDK/1AZQqXoQW1XJRZlgp9FLXwwB48uQJ165dw9XVlS+//JJx48bRpUsXXF2zl53hZfVI1q9fD2Rdj8Td3R03NzdAW49k9OjR+Pv7o6+vz9WrV1PaP69HAqSrR7J///6UdtmpR7J582ZmzpwJkFKPJLNFo8/rkQAp9UjGjBmTUo8kPDw8V/VI4P+//4CAgJR6JACxsbEpKeIXLVqU4XYz0759e/z8/GjatCklS5akSZMmGBjo5le+CiSvW63uUKq2djX8il7a6ygtx4GefpZdn9eJb1K2CV8f/povDn5Bl0pd+NF9LEOXXGTUijOs/aAJDuWyV5xIeXXKfP31a92/VPVICn09EtBOtU2YMAGAfv366SwfmLpGUhBYV4Fhe8C5Lxz8AVb0hJj72e7+Yp34QTvd+bQLmBc1ZLC3H7ei1EVSRdUjedvqkSQlJREZGQlAQEAAAQEBKWdv+U2dkRQURibw3jyo2AS2fQnzXaG3t3bqKxue14lvYduC8UfG89XRj+jcpDfbDtZh0NJTrB/ZFMtiRro9BqVAS12PpFOnTin1MABMTU1ZsWIFQUFBjB07Fj09PQwNDfnjjz+A/69HYmNjk2b66LnU9Ug0Gk1KzfUpU6YwePBgnJycMDExybQeybVr15BS0rZt25R6JD179sTHx4fWrVvnqR5JeHh4pvVIxowZg5OTE1JK7OzsMrwo/9zzeiRBQUH069cvXT0SCwuLDOuReHp6YmmZNglr+/btCQwMTPf9p65HotFoMDQ0ZO7cuVSsWDFNzXYvLy/c3d1ZvHgxFSpUwMfHB9DWI5k/fz6LFi0iISEhZWrSzMyMFStW6GxqSyVtLIjCzmunuh6GQrtvoPEo7Ur5bIpLjGPW2VmsDFxJmaLluXnlPRxKOrByWCOMDbOeMlN0QyVtfHVUPZKcU0kbCxsbZxhxEKp2gJ3jwWcQxGX//m9jA2O8GnqxsP1CpIjHuOJcLj5dx8er/UjSFP4/HBTwfMXVAAAgAElEQVTlVVH1SLTUGUlBJiUc+12bRdjSDvosh9K1c7SJR/GPmH5yOltvbCUp1paOpT/j5/faqQqLr0FhOSNR9UgKJ1WPJAu5DSRJjx6hV7z46/+l+98x8BkMcQ+hyy/g0i/Hm9gVsovxh6bwLCmWFtaezOnyMXpCnZC+SoUlkCiFU4Gd2hJCdBRC/CuECBJCeGXwuacQIkII4Z/8GJbqsx+FEJeEEIFCiNlCq3iqtv5CiPtCiFm6Gv/N4cMJce/Dw82bkfHZW32uExWbahM/2taHv0fB5o8hIS5Hm2hv157tPTdRQr82h6MW0WPDIO7GZF0SWMlfb8MfbsqbJ6//LnUWSIQQ+sBcoBNQC+grhKiVQdO1UkqX5Mei5L5NgWaAE+AANABaSikfp2rrAvwHZD9hVQ5IjQbz7t3RPHnCna/Gca1tWyLmzCUxIkIXu8ta8dLg8Te4fgFn/4LF7SDqRo42UapYSXb29cYmwYPrjy7TbeN7L60Tr+QvY2NjIiMj1fetFChSSiIjI9Pd1ZYTOpvaEkI0AaZIKTskvx4PIKWcnqqNJ1BfSjk6g75zgOaAAA4BHlLKwFRtqgL7gAoyi4PIyzUSqdEQc/QYUcv/IubQYTA0xPzdTlgO8KCoo0OutplnV3fChhHaayg9/oAanXPU/WFsAj0WbCbC2BuMQ15aJ17JPwkJCYSGhhIXl7OzSUXRNWNjY2xtbTE0NEzz/mu/RiKE6AV0lFIOS37tATRKHTSSA8l0IAK4CnwmpbyV/NlMYBjaQDJHSjnhhe1PAsyklF9msv8RwAiAChUq1MtoUU9OPQsOJnrlKh5u2IDm6VOKurhgNdCD4u3aIV74H6Bz0f9p7+a6cw6afgJtJ4N+9u8Rv/Mglh7zjpBguh8sd2BmZJZpnXhFUd5OBeEaSUZXqF+MWlsAOymlE7AHWAYghKgC1ARsgXJAGyHEi7/h3gdWZ7ZzKeUCKWV9KWX9kiXzJ99UEXt7yvxvAlUOHaT011+TGB3F7c+/IKjtO9yfP5/E5NW8r4RlRRiyExoMg2OzYVlXeJz9ax5lLYqybEgjEiJbYh79JRZF/r9OfEyCKt2rKEr26TKQhALlU722Be6kbiCljJRSPr+PcCFQL/l5D+CElPKJlPIJsB1o/LyfEMIZMJBSntHV4F9G39QUq4EeVN6+Hdv5f1CkalUiZv1GUKvW3Pl6AnGBgVlvJD8YFIHOP4PbIgjz166GDz6U7e41ypjx58B6hIZbYBg+hkG1BrPh2gZ6be6F/z1/HQ5cUZTCRJeBxA+oKoSwF0IYoT2D2Jy6gRDCJtXLbsDz38A3gZZCCAMhhCHQMtVnAH15ydnIqyL09CjeqhUVFi+i0tYtmPd049H27QT3cOO/AR482rkLmQ9FbLLk1BuG74eiFvBXdzj8s7ZOfDY0rWzNzN7O+AU/JvhaK5Z0WIpEMmjHIGafnU2CJkHHg1cU5U2n03UkQoh3gVmAPrBESvmdEGIa2hz3m4UQ09EGkEQgChglpbySfMfXPKAF2umwHVLKz1Nt9wbwrpTyCtnwKhckJj18yIP1G4heuZKE27cxKGuDVb9+WPTqhb6FhW53/uwJbPkELq7XrorvMR9MrLLV9c+D15m+/QrDmtszpn0FfvD7gb+D/qZWiVpMbz6dShaVdDt2RVEKnNd+sb0geR0r22VSEk8OHCDqr+U8PXkSYWyMedeuWHoMwLhaNR3uWILfItgxHsxsoPcyKFc3G90kU7dcxvtYCB+0qMQX7atz6PY+ph6fSmxiLJ/X+5y+Nfq+/sWZiqK8MiqQpPK6U6TE/XuV6BUrtAsbnz3DpHFjrDwGYNqqFUJfR0kUQ89o7+p6Eg4dZ0D9IVkmfkzSSCZuusiqkzepXdaMWX1csDR7xsSjEzly+wjNyjZjWrNplDLJuPaBoiiFiwokqbzuQPJcYnQ0D3x9iV61msSwMAxtbbHs3x+Lnm7ov1CjIV88jdKuNwnaDY7u0HUWGGWdjnvnpbuM33CBJ88S8epYg0FNKuJ7zYeZp2dSxKAIkxpPor2dbuoaKIpScKhAkkpBCSTPycREHu/ZS9SK5cSePoMwMcHive5YDhhAkUr5fC1Co4EjP8P+78G6Grgvh5JZT61FPNaW69175R7Nq1jzU28n4rjL14e/5mLkRbpV7oZXQy+KGxXP3/EqilJgqECSSkELJKnFXb5M1PIVPNq6FZmQQLFmzbAa6EExV1eEXj7eVHfjAPgOhYRY6DYbHLOu0yClZI3fLb7ZehkDPcG3PRzp5FiSBQELWBiwkNImpfmu+XfUL5PlvzNFUd5AKpCkUpADyXOJkZE88PHRTnvdu4dhxQpY9R+AuVsP9E1N82cnj+5oswjfOgENR0D7b7VrUbIQcj+Gz9b5c+7mA7o5l+Wb7g6ExFzm68Nfc+vxLTwdPBntMhojfVWBUVEKExVIUnkTAslzMiGBR7t2Eb18BbH+/ugVK4a5mxtW/fthZGeX9x0kJWjrmxyfA+Xqacv5WlTIsltikob5B68za881rE2LMLO3M3XtTPjp9E/4XvWlumV1prtOp6rl21vcR1EKGxVIUnmTAklqsRcuELV8OY+274DERExbtMDSw4NizZrm/Tbcy5th00egp69dGV/1nWx1uxD6kDFrz3E9IobBzewY17EGJ+4eZvKxyTyJf8KndT9lQK0BqtaJohQCKpCk8qYGkucSIyKIXrOW6LVrSbp/H6NKlbDyGIB5t27oFcv6LqxMRV6HdYMg/CK0GAutvLSBJQux8Un8sOMK3sdCqFrKlF/7uGBjlciU41M4cOsAjco04tvm31KmWJncj01RlNdOBZJU3vRA8pwmPp7HO3YQ9ddy4i5eRK94cSx69sRyQH+MbG1zt9GEWPjnSzi3AuxbQs/FYJq9JJeHrkYw1vc8UTHxjHmnGh+0qMSm6xv5we8HDPQM+F+j//FupXdzNy5FUV47FUhSKSyB5DkpJbH+/kQvX8GjXbsgKQnTNm2w8hiASaNGuZv2OrtcG1CKWoL7X1C+Yba6PXgaz4SNF9l2IYz6FS35xd0FYRjJ+CPjOR9xnk52nZjQeALmRcxzPiZFUV6rfAskQojKQKiU8pkQohXaqoV/SSkf5MtIX4HCFkhSSwgPJ3r1ah6sXUdSdDRFqlbF0mMA5l27ole0aM42dvcCrPWAR7eh2+/g/H62ukkp2eR/h4mbLqLRSCZ1rYVbXRuWXFzC/PPzKVG0BN82/5bGNo2z3piiKAVGfgYSf6A+YAfsRJvBt7qU8o2ZsyjMgeQ5zbNnPNq6jajly3l25Qr65uZYuPfGsm9fDMuWzf6GnkZpU6sEH4JmY6DtpGxdNwG4/SCWL9ed5/iNSNrVKs10N0fC44LwOuxFyKMQPGp58GndTymin/Utx4qivH75GUjOSinrCiHGAnFSyt+FEOeklHXya7C69jYEkueklMSeOUPUX8t5vGcPAMXfeQergR4UrVcve9NeSQmw/Ss4vQSqdYKeC6FI9lawazSSJUeD+XHnv5gZGzDDzYlm1cz45fQvrPl3DVUsqjDddTo1rGrk5TAVRXkF8jOQnESbCn4C0FVKGSyEuCilfE0Fy3PubQokqSXcuUP06tVEr/NB8/AhRWrWxGrAAMy6dEavSDbOCk4thO3joGQN6LtaW5Uxm/69+5gxa/0JDHtE34bl+V/nWpy7f4JJRycR/Sya0S6j8aztiX42z3YURXn18jOQ1AJGAsellKuFEPZAHynljPwZqu69rYHkOU1sLA+3bCF6+QqeXbuGvqUlFn3ctdNepUu/vPP1feDjCXqG0GcFVGyS7f0+S0zi193X+PPQdSpYmfCLuwuVSwumnZjG7v92U7dUXb53/Z5ypuXydoCKouiETu7aEkJYAuWllAF5Gdyr9rYHkueklDw9eZKo5St4sm8f6Otj1r49lh4DKOrikvm01/0gWN0Hov+Drr9Bnf452u+p4Cg+X+fPnQexfNS6Ch+3qcL2kK1MPzUdgPENx9OtcjdV60RRCpj8PCM5gLaKoQHgD0QAB1NXLCzoVCBJL/7WLaJXruLB+vVoHj/G2MEBq4EeFO/YET2jDHJmxUZrz0xuHIAmo6HdtGxfhAd4HJfAtC2X8TkTimM5c37t40xRk0d8ffhrzt47S7uK7ZjUeBIWxjquIqkoSrblZyA5J6WsI4QYhvZsZLIQIkBK6ZRfg9U1FUgyp4mJ4eHmzUQtX0H8jRvoW1tT6osvsOjxXvrGSYmwczycWqAt5dtzERjnrI7KjothjN9wgafxSYzvVIP+jcqz/Mpf/H7udyyLWDKt2TSal2ueT0enKEpeZDeQZCchkoEQwgZwB7bmeWRKgaJXrBiWfftSaesWyi9ahFHFioSNH8/d779HJiambaxvAO/+BJ1/gaA9sLg9RAXnaH8dHWzY+VkLmlYuwZQtlxmy7Azvlu/H6s6rMS9izqg9o/juxHfEJsbm41EqiqJL2Qkk09CuH7kupfQTQlQCrul2WMqrJvT0MG3ejIrLvLEaNJDov5Zzc/hwEqOj0zduMBQ8NsLjMFjYBkKO5GhfpYobs8SzAd/1cOB0SDQdZh0iKNSMNV3WMKDmANb8uwb3Le5cun8pn45OURRdUilSlAw9WL+Bu1OmYFCmDOXnzaVI1QzSw0deh1V9IDpYe5ZSb1CO9xN8P4bP1vrjf+sB77mUZWp3BwIfnGHCkQlExUYx0nkkQx2HYqBnkA9HpShKTuTb1JYQwlYIsVEIcU8IES6EWC+EyGWGQOVNYdHTjQp/LUMTF0tIn/d5vHdv+kYlKsOwPWDfArZ8AjvGa6+j5IC9dTF8Rzbh83bV2BIQRsdZh9DEVGFDtw20q9iOOf5z8Nzhya1Ht/LpyBRFyW/ZmdpaijYtSlmgHLAl+b0sCSE6CiH+FUIECSG8MvjcUwgRIYTwT34MS/XZj0KIS0KIQCHEbJF8b6gQwkgIsUAIcVUIcUUI0TM7Y1FyzqROHex9fTGqXJnQj0YTMW8e6c5gi1pAPx9oNBJOzNPeJhz3MEf7MdDX45O2VdkwqilFDfXpt+gks3ffZlrT6cxwncGNBzfouaUn66+uT79/RVFeu+wEkpJSyqVSysTkhzeQZZ5xIYQ+MBfoBNQC+iYvbnzRWimlS/JjUXLfpkAztAkiHYAGQMvk9hOAe1LKasnbPZiNY1ByybB0aSou/wuzbl25P/t3bo/5DM3Tp2kb6RtApx+gyyzt7cGL3tFOe+WQc3kLtn3iysAmFVl8JJhuc45gZ9ycDd034GjtyJTjU/hk/ydExkbmz8EpipIvshNI7gshBggh9JMfA4Ds/CQ3BIKklDeklPHAGqB7NsclAWPACCgCGALhyZ8NAaYDSCk1Usr72dymkkt6xsaU/eEHSo0dy+Pduwnp15+E27fTN6w/GDz+hpgIWNRWm/gxh4oa6TOtuwPegxsQ/TSB9+YeZaNfDPPfWcDY+mM5dvsYbpvdOHDrQN4PTFGUfJGdQDIE7a2/d4EwoBcwOBv9ygGpJ7ZDk997UU8hRIAQwlcIUR5ASnkc2J+8vzBgp5QyUAjxfLXaN0KIs0IIHyFEhjk+hBAjhBCnhRCnIyIisjFc5WWEEJQYOoTyf84n4fZtgnv15qmfX/qG9q4wfB8UKwXLe8DpbM2CptOqeil2jWnBOzVL88OOK/RbeJLWNr1Y02UN1kWt+Xjfx0w9PpWnCU+z3piiKDqVZSCRUt6UUnaTUpaUUpaSUr4HuGVj2xnlu3hxgnsLYJe8uHEPsAxACFEFqAnYog0+bYQQLdCurrcFjkop6wLHgZmZjHuBlLK+lLJ+yZLZq/inZM3U1RW7tWvRt7Dgv8FDiF6zJn0jq0owbDdUag1bx8A/X+X4IjyAZTEj5vWvyy/uzlwJe0zHWYc4d92YVe+uYnDtway/up7eW3pzPuJ8PhyZoii5lZ0zkoxkJz1KKFA+1Wtb4E7qBlLKSCnls+SXC4F6yc97ACeklE+klE+A7UBjtFNqT4GNye18gLq5OgIl14pUssdu3VqKNWvK3SlTCZs8BRkfn7aRsTn0W6tNp3LqT1jVG2JzXgtNCIFbXVu2j3HFoZw5X/kG8MnqCwyq8RGLOywmQZPAoO2DmOs/lwRNQj4doaIoOZHbQJKd7Hp+QFUhhL0Qwgh4H+3dX/+/Ee2K+ee6AYHJz28CLYUQBkIIQ7QX2gOl9padLUCr5HZtgcu5PAYlD/SLF6f8vHmUGD6MB2vX8t+QISRGvnDpTE8fOnynrbYYfDjXF+EBbC1NWDW8MV+/W4P9VyLoMOswjx9UYH239bxr/y7zz89n4D8DCX6Ys5X2iqLkXa4WJAohbkopK2Sj3btoa5noA0uklN8JIaYBp6WUm4UQ09EGkEQgChglpbySfMfXPKAF2umwHc+TRAohKgLLAQu0CSQHSylvvmwcakGibj3cspWw//0P/RJWlJ87F+OaNdM3CjkK6zxAkwTuy6BSq1zvLzDsEZ+t9efK3cf0a1SB/3WuyeE7e/nmxDc8S3zGl/W/xL26u8omrCh5lOekjUKIx6S/pgHas5GiUso3ZqmxCiS6F3vxEqGjR5P08CFlp3+PWceO6RtFh8Cq9+H+Ve3twg2H53p/zxKT+GXXVRYcvkFFKxN+7eNCOesEJh6dyLE7x2herjnfNPsG66LWuT8oRXnL5Xllu5SyuJTSLINH8TcpiCivRlGH2tj7rMO4Rg1uj/mMe7/9htRo0jaytIOhu6BqO/jnS9j2hbasby4UMdBn/Ls1WT28MQlJkl7zj7Pi6ANmt56LV0Mv/O760WNTD/b+l8GKfEVR8pXKtaXkK018PHenTeOh73pM27Sh7I8/oG9q+kKjJNgzBY7NBvuW2qmuopa53uejuASmbr7M+rOhONma82sfF4ThPbwOexEYFch7Vd5jXINxmBqZZr0xRVFS5GcaeUXJNj0jI2y++YbSEybw5OBBQt5/n/ibL1zC0tOH9t9A93lw8zgsbAv3c59Q2szYkJ/dnfmjf11uRj2l8+zDHAnUY0WnFQx3HM7m65vptaUXZ8PP5vHoFEXJiAokSr4TQmDlMYAKixaSFHGf4N7uxBw7lr5hnf4waIs2N9fCthCUt2moTo427BrTgkb2JZi06RLD/vKnT5UReHf0RiDw3OHJrDOzSMjldJqiKBlTgUTRmWJNmmDn64NhqZLcHDacqGXL0iddrNAYRuwHi/Kwsjec/BPyMN1ayswY78EN+OY9B04FR9Jh1iHCwsvg282X96q8x+KLi+n/T3+uP8jdbciKoqSXnTTyj4UQj1543EpOLV/pVQxSeXMZlS9PxdVrMG3TmvDpMwj7egKaFxcvWlSAITuhWkfY/hVs/SzXF+FBe0bk0bgi2z5xpaKVCR+uPMvEDdf4st5EZrWexd2Yu/TZ2oeVgSvRSE3WG1QU5aWyU7N9KtoV6avQ3vr7PlAG+Bftuo9WOh5jnqmL7a+f1Gi4P2cu9+fNo6izM+V+n41hqVJpG2k0sG8aHPkV7FzB/S8wscrTfhOSNMzZF8Sc/UGUMTNmZm9nqpaVTD42mUOhh2hi04Rvmn1D6WIZpmxTlLdanteRpNrQSSlloxfeOyGlbCyEOC+ldM7jWHVOBZKC49HOXdzx8kLfzAzbOb9T1NExfaPza2Dzx2BWTptmpWT1PO/33M1oPl93npDIGIY1t9cW0grewMzTMzHUM+STOp/gVs0NQz3DPO9LUQqL/LxrSyOEcBdC6CU/3FN9VvjvHVbylVmH9titWY0wMOC//gN4uHlz+kbO74PnNoh/ok2rcm1Pnvdbp4Il2z5pTr+GFVh4OJge847haNaRdV3WUdWyKt+e/JYem3qwM2SnKp6lKDmUnUDSH/AA7iU/PIABQoiiwGgdjk0ppIyrV8fO14eiLi7c+Woc4T/+hExKStuofEMYvh8sKmoTPh6fl6eL8AAmRgZ818ORpZ4NuP8knu5zjrLTP4lF7ZYwp80cDPUM+fLgl/Td1peTYSfztC9FeZuoBYnKayMTEgifPoPoVaso5upKuZ9nom9mlrbRsyew8QO4shXqDoR3fwYDozzvOyomnvEbAth5KZyG9lZM7lqLGmVM2XpjK3P853A35i5NyzZlTN0x1CyRQe4wRXkL5Oc1Elvgd7SlbyVwBPhUShmaHwN9FVQgKdii167j7jffYGRri+28eRSpZJ+2gUYD+7+DwzOhYjNwXw7FSuR5v1JK1p+9zdTNl3j8LJGG9lYMaWZPi+oW+Fxdy8ILC3n47CHv2r/L6DqjKV+8fNYbVZRCJD8DyW60d2wtT35rANBfStkuz6N8RVQgKfienj5N6CefIuPjKffzTExbtkzfKMAHNn0ExctoL8KXyp8zhYexCazzu4X3sRBuP4jF1rIonk3t6ORsgW/QclZcXkGiTMS9mjsjnEZQomjeg5iivAnyM5D4SyldsnqvIFOB5M2QcOcOt0aP5lngFUp98TlWQ4emTwUfehrW9IP4p9BrMVTrkG/7T0zSsCcwnCVHQjgVEkUxI3161y9Pl7rF2HZrGRuvbaSIfhE8a3sysPZAihkWy7d9K0pBlJ+BZA/gDaxOfqsv2hogbfM6yFdFBZI3hyY2lrAJE3j0z3bMunTB5ttv0DM2Ttvo4W1Y0xfCAqDdNGj6MeRz7ZGLtx+y5GgwW87fIVEjaVO9FO/W1edI5HL23NyDlbEVI51H0qtqLwz11S3DSuGUn4GkAjAHaIL2Gskx4JOsikkVJCqQvFmklEQuWEjErFkY16qF7dw5GJYpk7ZRfAz8PQoubwKXAdDlFzAoku9jufc4jpUnbrLixH9ExsRTvXRx2teNI+DpKs7eO0P54uX5uM7HdLDrgJ5QGYeUwiXfAkkmGx8jpZyVq5G9BiqQvJke79vPnbFjEUWLYjt7NiZ166RtoNHAwRlw8Aeo0ER7Ed60pE7GEpeQxJbzd1hyNITAsEdYmBjQyiWKG0nrCH4URE2rmnxW7zOalG2ik/0ryuug60CSrVK7BYUKJG+uZ0FB3ProIxLuhGEzZTIWPXumb3RxPfz9IRQrBf3WQOnaOhuPlJKTwVEsORLM7sBw9IWkbu0QIgw2cT/uLo1tGjOm3hhql9DdGBTlVdF1ILklpXxj7oVUgeTNlvTwIbc//4KYo0ex9PCg9LivEAYvFOm8fVZ7Ef7ZY3BbCDXe1fm4bkY+ZdnxENb63eJJfBz29ud5arKTp0mP6GjXkY/rfEwFszfm7y1FSUedkaSiAsmbTyYmcm/mz0R5e2PSuDHlfv0FA8sXqio+uqMNJnf84Z3J0GxMvl+Ez8jjuAR8z4TifSyE/6KjsCp7DGl2EEkivar1YqTzSFU7Xnkj5TmQCCEek3EuLQEUfZPqtqtAUng82Pg3dydNwqBMGWznzsG4WrW0DeKfateaXNoAzn2hyywwNM54Y/ksSSPZf+UeS44Gc/y/YIqW2o+B+UntLcMOgxhUa5Aq96u8UXR6RvKmUYGkcIk9f57Q0R+jiYmh7I8/UPydd9I2kBIO/aRdDW/bEN5fCaalMt6Yjly5+4ilR0L4+5I/wmoHhmYXKGZgzkcuI+lTwx0j/byneVEUXSsQgUQI0RH4DdAHFkkpZ7zwuSfwE3A7+a05UspFyZ/9CHRGm1hyN9q0LFIIcQCwAWKT+7SXUt572ThUICl8EsLvEfrxx8QFBGD98WisR41C6L1w++2lv2HjSChmDX1XQ5kMUtbrWOSTZ6w+dRPv04eJMd2EQbEbmBuW5ov6n9K9amd1y7BSoL32QCKE0AeuAu2AUMAP6CulvJyqjSdQX0o5+oW+TdEGmBbJbx0BxkspDyQHki+llNmODCqQFE6aZ8+4O2kSDzdtpnj79pSdMR09E5O0je74w+q+2rrwbgugZpfXMtb4RA3/XLjDnBP/cEfPF33jMCwN7PiiwWd0q9o6/Qp+RSkA8rMeSW41BIKklDeklPHAGqB7NvtKwBgwAooAhkC4TkapvLH0ihTBZsYMSo0bx+M9ewjp24/40NtpG5V10daEL1UD1vaHwz/nOR19bhgZ6PFeHVt2jxzOsg6rqKE3ksinj/jf8U9xXd6HdQFHVR0U5Y2ly0BSDriV6nVo8nsv6imECBBC+AohygNIKY8D+4Gw5MdOKWVgqj5LhRD+QoiJIpM/5YQQI4QQp4UQpyMiIvLlgJSCRwhBicGelP/zTxLCwgjp3ZuYU6fSNipeRlsoy6EX7J0GG0ZAQtxrG28DO2t8PD5iW88t1DMdzMPEm3xzbiRNFnuy6MRJ4hNVHXnlzaLLQJLRL/gX/+TaAthJKZ2APcAyACFEFaAmYIs2+LQRQjyf5uovpXQEXJMfHhntXEq5QEpZX0pZv2RJ3ax2VgoOU9fm2K9bi76VFTeHDCV69eq0DQyLQs9F0OZ/cGEdeHeGx6/3JNfOygzvnp+zr89OmpXoy1ODi8y6MoKGf45ixq6T3H/y7LWOT1GyS5eBJBRIvWjRFriTuoGUMlJK+fynZSFQL/l5D+CElPKJlPIJsB1onNzndvJ/H6NNb99QZ0egvFGM7OywW7sG0+bNuTt1GmGTpyDj4/+/gRDQYqw2lcq9y7CwNYSdf30DTlaymDnzu3zN3t47aVGmK0nFTrLi9ihcF4zjM5/jBIY9et1DVJSX0mUg8QOqCiHshRBGwPtAmgLdQgibVC+7Ac+nr24CLYUQBkIIQ6AlEJj82jq5ryHQBbiow2NQ3jD6pqbYzp1DiREjeLB2Lf8NGUJiZGTaRrW6wZCdgID/a+/O46Oq7/2Pvz6Zyb5OEraQjUVFBRQERPYtELVi3dcCrRu0Xr1d7q/t7/Z3e3/trd7b1urtT9xQL4gLLq2KYhOCCwiioiyyKglJIAlbMpOE7JmZ7++PM4QhBgkkk0nI5/l45MGcmXPOfL4MmTff8z3ne57PtiZ+7Ab6RDXr498AACAASURBVCfzRPZ/sOr6lUweOBV70vvkHftn5r7wMLc88zF5uw7j8eo4iup+An3671XAY1in/z5vjPmDiPwO+MIYs1JEHsYKEDfgBBYZY/b4zvh6AuusLQPkGGN+JiLRwDqswXcb1uGwnxljPN96cz961lbvVLVqFQf/9TfYEh2kLV5MxIWtboR17LA1AF+yCZLOg6QhkDQUEgefeBybAq1PK+4iOyt28simR9l0+DPE7aDucBYpoRNYcMVgbhqTSmyETl+vAivop/92JxokvVf9zp2U/OR+PJWVpDz8EHFXXnnyCs0NsPFx6xBXRQE494G7/sTr9khfsAyGRF+4HA+Z6D5dMgXLxrKNPPrlY+x27iLMm0plySwi3Rdz85h0FkzIJD0p6vQ7UeosaJD40SDp3dzl5ZQ88CD1mzeTtPA++jzwwLcvXjzO64VjZb5QKbD+rCiAinxwFYG3+cS6YbG+UBnSKmSGQKSj7f2fJa/xsrpoNX/d8lcOHDtAggzjUNFM3PVpzLqwHz+aOIjxgxP1ehTVqTRI/GiQKNPUxKHf/weVr79OzPTppPzpj9hiznDeK48bqvZDxT5fyOSfCJmqA2D8TtuNTDwRLIlDTg6c8LOfb6vZ08zf9v6NJ7c9ibPBSXr45ZQVTsdVlcCw/rH8aNIg5l6SQkSo7azfQ6njNEj8aJAosO4l4nr5ZQ4/9DBhgzJJW7yYsIyMztm5u9HqsbT0ZPJP9GaOlZ28bkz/tnsyjkHtnmCyrrmOZbuWsXTHUho9jYyMz6KscDL5h2wkRYdxx+Xp3Dk+g75xXTNhpTo3aZD40SBR/mo/+5zSBx/EGMPAvzxCzMSJgX3DplpwFlrh0vpwWV2534oC8WnWeEzS0JNDJiEd2rg3fEV9BUu2L+HVr1/FLnamDbieoyUTWLunBnuIcM3IFH44cRAjUuMD20Z1TtIg8aNBolprKimh5Mc/oTE/n36//F845s0LzvhCQ9WJYGndk2msOrFeiB0SMlqdWeYLmbhUDtSWsnjrYlbtW0V8eDw3DJ6H8+BY/v7lYWqbPIzNdPDDiYOYfVE/7DadKFK1jwaJHw0S1RZvbS1lv/o1x/LyiBgxgsQF84mbPRsJ7Qan1RoDdRUnei4tIeMbn2muO7GuLbzllOU9cUk8Vl/Ihpoi+kf24a4RP6H66EiWf3qAA856BiZEMn9CBreMSSc+qhu0U3VrGiR+NEjUqRivl8o33sD5/P/QVFSEvX9/En9wJwk33YQtLi7Y5bXNGDh2sFXI+H5cheBp4vOIcB5NTGBHeDhDvSE8EDGINLmA3IMxvH8khlJ7KkPS0xiVnsDodAej0h0kRus9UtTJNEj8aJCo0zFeLzVr1+Jc9gJ1n36KREWRcP31JM77AWHpPeau0uD1WGeQVRRgyvPJO7iBv1bvoJhmRjc08lOni0sbrWljPgm7gv+q/R7bPIMAGJQczai0BEZlOBidnsAF/WL1MFgvp0HiR4NEnYmG3btxLnuBqlWrwO0mZuYMkubPJ3LMmB55nUazt5k3977Jk1ufoLyhghmOi3kwNIXBW9+AxioqB07lg74L+EdVOlv2uyivsYImKszGyNR4Rqc7fL2WBJJiwoPcGtWVNEj8aJCos9F85AiuV16h8pUVeCoribj4YmscJTu7e4yjnKG65jpe3P0iz+94nrrmOsb2HcUcTxhZu9/HUVsBmZMxU35BSfxYNh+oZHOxiy0HKtlVVo3bN8dXRlKUL1gSGJXuYFh/7bWcyzRI/GiQqI7w1tdTtfIdnMuW0bRvH/Z+/XDccQeOm2/ClpAQ7PLOmKvBxSt7XuEfhf+gqLoIm9gYH5nCnEOFzHAdJD5ljDVL8nmzQYT6Jg/bS6vYst/F5v0uNu+v5Ogxa9LuyFBfryXDwai0BEZnOEjWXss5Q4PEjwaJ6gzG66V2/XqcS5dS+8lGJDKShOu+j+MHPyB80KBgl3fGjDF87fqanMIccopyKK0pxU4IE5s8zKmsYHrsYGKm/C8Yds1JE1caYyhx1bPleK9lv4udfr2W9MSolkH80ekOhg2IJVR7LT2SBokfDRLV2Rq+/gbnsmVUv/MOxu0mZto0EufPJ+rycT1yHMUYw86KneQU5pBblMuhukOEGZhcV0d2SDxTxv2UqEtuA5u9ze0bmj3sKK2yeizFlWze7+KIr9cSERrCyIEJjMo4ES59YrXX0hNokPjRIFGB4i4vx/XyK7hWrMDjdBJ+4YUkLZhP3JVXImE983Rar/Hy1dGvyCl8j9UF73C0uYZIr5ep7hCyh17LpAm/JDw89jv3YYyhrKrB12OxgmVnWRXNHuv7JtURedJYy0Upcdpr6YY0SPxokKhA8zY0UPWObxwlvwB7nz447ridhFtuwe7o3JmAu5LH62HzoS/I2baEvMOf4xJDtNcwPWEYV466jyvSphHaxtQtbWlo9rCzrKolWDYXV3KougGAcHsII1PjGeULl9HpDp0nrBvQIPGjQaK6ijGG2vUbcC5bRu369UhEBPHXXkvi/HmEDx4c7PI6xO1p5vMtz5C7/QXWmGNU22zEhYQzMyOL7CHXMG7AOOwhbR/6OpWyyvoTwbLfxc7Sapo81izKAxMiT4y1ZDi4aEAcYXbttXQlDRI/GiQqGBr37sX5wgtUvb0S09RE9NQpJC1YQNT48T1yHKWFMTQXrmXjx38g51gBH0RHURsiOMLjmZUxm+zMbC7rdxm2kDOfyr7R7WFnWXXLIbEt+12UVVm9ljB7CCMGxrf0WEZnOOinvZaA0iDxo0GigsldUYFrxQpcL7+Cp6KC8AsuIHH+fOK+dzUhPXQcpUXJFzSu/SPrS9eRGxvPR9GR1BsPyZHJZGVkceWgK7mkzyWEyNn3JA5VNbB5v8t3+nEl20uraHJbvZaU+AjflfjWBZMXp8QRbtd7sXQWDRI/GiSqO/A2NlL97iqcy5bR+M032JKTcdx+G45bb8WemBjs8jrm4Ffw8SPU717Jutg4clLO5+NmJ43eJvpF9WNO5hyyM7MZnjy8w72xRreHXWXVLYfEtuyvpLTSuj1ymD2E4SlxLfOHjc5IYEB8ZGe0sFfSIPGjQaK6E2MMdRs3UrFsGbVr1yHh4cTPnWuNowwdGuzyOuboN7D+L/DVa9Ta7Hx44Uxyo6NYf3Qzbq+bgTEDW0JlWOKwTjvEd7i6oaXHsrnYxfbSKhp9vZYB8RGMTncwJtPB9aNSddbjM6BB4keDRHVXjQUFOF9YTtVbb2EaG4meNInEBQuInjihZ4+jOAthw2Ow5SXAUD3iBj7IHENOxWY+LfsUj/GQGZfZEipDHZ0boE1uL7sPVrdcib9lv4sSVz0x4XbuuDyduyYN0rPC2kGDxI8Gieru3C4Xla++ivOll/AcLSf8vKHWOMo11xAS3oMv3qsqhU/+Cl8uBU8TXHwdrsvvYU19CbmFuWw6vAmv8TI0YWhLqGTGZwaklF1l1Ty1toB3vyrDHhLCDZelsnDqYDKSogPyfueCbhEkIpIN/DdgA541xvxnq9cXAH8CSn1PPW6Medb32h+Bq4EQIA940PgVKyIrgcHGmOGnq0ODRPUU3qYmqt97D+fSZTTu2YMtMRHHbbfhuO1W7MnJwS7v7NUcgY2LYdOz0FQDF1wNU35OeWIGecV55BTmsPnIZgCGJQ4jOzObOZlzSI1N7fRSiitqeXrdPt74ogS318vVI1NYOHUwF6fo7YhbC3qQiIgN+AbIAkqATcBtxphdfussAMYYY+5vte0ErICZ4ntqPfBrY8xHvtevB24ERmqQqHORMYa6zz7HuXQpNR99hISGEjf3GhLnzyfi/PODXd7Zq3PCZ0/DZ09atxkeMtOaIDLjCg7VHmJ10Wpyi3L5qvwrAEYkj2BO5hzmZM6hf3T/Ti3lSHUDz20o5KVP91PT6GbaBX1YNHUI4wYl9uzDip2oOwTJFcC/G2Pm+JZ/DWCMedhvnQW0HSRXAI8DkwAB1gE/MMbsFpEYIAe4F3hNg0Sd6xr3FeJc/gJVb76FaWggesIEEhfMJ3rSJCSkh16g11ANXzwHnzwOdeWQMQmm/AIGTwMRSmtKyS3KJacwh93O3QCM6juK7MxsZmfOJjmy83pnVfXNvPhpMc+vL6SitonLMhz8eNoQZgzr2+sDpTsEyY1AtjHmbt/yD4DL/UPDFyQPA0exei8/NcYc8L32Z+BurCB53Bjzr77nH8UKli3Au6cKEhG5FytsSE9Pv6y4uDgQzVSqy7hdLipfex3XSy/hPnKEsCFDSJw3j/hr5xIS0UMHjpvqYPMy2PDf1u2DB15m9VDOzwbfl3hxdbEVKkU57HXtJURCGNNvDHMy55CVkYUjonOmoKlv8vD6lwd4eu0+SivruaBfLIumDeF7Iwf02nuudIcguQmY0ypIxhlj/slvnSSgxhjTKCILgZuNMTNEZCjW2MotvlXzgF8C1cDvjTHXiEgm3xEk/rRHos4lpqmJ6pwcnEuX0bBrFzaHg4RbbyHx9tux9+kT7PLOjrsRtr4E6x+Fyv3QbwRM+TlcOBf8rpAvqCwgpyiHnMKclnupXD7gcrIzs5mRPoP48I6PczR7vLyzrYwnPypg75EaUh2R3DdlMDeNSSMitHdd7NgdguS0h7ZarW8DnMaYeBH5FyDCGPN732v/BjQAx4D/AzQBdqAv8IkxZtp31aJBos5FxhjqNm3CuewFaj74AOx24q++msQF84kYNizY5Z0dTzNsfwM+fgQq9kLy+TD55zD8xpOmsDfG8I3rm5ZQKakpwR5iZ2LKROZkzmF62nRiwmI6VIrXa3h/zxGe+CifLfsrSY4J50eTMrlzfAZxEb3jWpTuECR2rMNVM7HOytoE3G6M2em3zgBjzEHf4+uAXxpjxovILcA9QDbWoa0c4DFjzDt+22aiPRKlAGgqLsb5wnIq//53TH09UePHkzh/HjFTp/bMcRSvB3a9bQXK4R2QkAGTfgqX3g72k0+HNsawq2IX/yj8B7nFuRyqPURYSBiTUyeTnZnNlNQpRIVGnXUpxhg+K3TyxEcFrPvmKLHhdu68IoMfTRx0zt9XJehB4iviKuAxrNN/nzfG/EFEfgd8YYxZKSIPA3MBN+AEFhlj9vh6J09gnbVlgBxjzM9a7TsTDRKlTuKpqqLy9ddxvvgS7kOHCMvMJHH+POKvvZaQqLP/Mg0aY+CbHFj3Jyj9EmJTYOKDMHoehH27PS33UinKYXXRao7WHyXSHsmU1ClkZ2YzaeAkIuxnP560o7SKJ9cW8N72g4TaQrh5TCr3TRlCWmIP/Ltth24RJN2FBonqbUxzM9W5q3EuXUrDjh3Y4uNJuOUWHHfcQWi/vsEu78wZA/s+hHV/huINEJUME+6HsXfDKW6y5fF62HxkM7lFueQV5+FscBJpj2TywMlkZWYxZeDZ91QKy2t5Zl0Bf/uyFI8xXDNyAAunDWFY/7iOtLLb0SDxo0GieitjDPWbN+Ncuoxja9aA3U7cldkkzp9P5MUXB7u8s1P8iRUoBe9DRAKMXwTj7oWoU0986fa62XRoE2uK1/D+/vepaKgg3BbOhJQJZGVkMTVtKnFhZx4Ch6oaeG79Pl76bD91TR5mDuvLj6cP4bKMHj4Jp48GiR8NEqWg6cABnMuXU/XG3/DW1RE9cSLJC+8jauzYYJd2dkq/hHWPwNerICwWxt4FV9wPMd995prH62Hr0a3kFeexpngNh+sOYw+xM37AeLIyspieNv2MTymurGvihY3F/M+GQlx1zYzLTGTR9CFMO79Pj74WRYPEjwaJUid4qqtxvfoqzqXL8FRUEDnmMpIXLuq5E0Ue3mkNyu/4O9gj4LIFMPEBiEs57aZe42V7+XbWFK8hrziP0ppSbGJjTP8xzM6YzYz0GWd08WNdk5tXNx1gybp9lFU1cOGAOBZNG8JVw/v3yGtRNEj8aJAo9W3e+noqX3+Diueew334MBHDh5O8aCEx06f3zDO9yvda16FsW2Fde3LpHTDpn8GR2a7NjTHsdu5uCZWi6iIEYVTfUczOnM3M9Jntnqalye3l7a2lPLW2gIKjtWQkRXHvlMHcMDq1R12LokHiR4NEqVPzNjVR9dZbVDyzhOaSEsLPP5+k++4lLjsbsfWcL70WrmLfFPYvWqcRj7wZJv0M+rR/jjJjDPmV+awpXsPq4tXkV+YDMLLPSLLSs5iVMatdE0p6vYbVuw7z5Ef5bCupok9sOHdPGsTtl6cT2wOuRdEg8aNBotTpGbeb6vfeo/ypp2nat4+wzEyS7r2X+Gu+h4R2/y+9b6kus+by+uJ5cDfA+XMgph/YQiEk1LrAMST0FMt2v+dDKWqqZE3lHla7drK7tgSAC2PSyeo3lln9L2dQbEab2x1fNiF2Nu6r5Im1+1ifX05chJ15V2Tyw4mZJMV032tRNEj8aJAo1X7G6+XY6jzKn36axt27CU1JIemeu4m//vqeeW+U2nJrCvudf4fmeuvqea/b96fv8Rkosdt4PyqK1dFRfBVh/X0MbWoiq7aerNo6hjY3c8qRppBQPCF2Gr0h1HtC8GAnLCyMmKhI7KFhpw20k5fbud7Yu6zHZ0GDxI8GiVJnzhhDzdq1VDz5FPXbtmHv04fEu36E4+abe+bFjadizMnB4nH7/mxr2X+9Zg7Vl/N++VbynNvZXFOMATLDHMyKHUJWzGAuDI1HvO5vbYfXTVVNHXvKnJRWVGMzHjISQhmaHE6MnVO836nqaFVva7858q3ZANpLg8SPBolSZ8+6N8pnlD/5FHWffYbN4SBx/nwcd9yOLbbtiwF7o/L6cj7Y/wF5xXlsOrQJj/EwMGYgs9JnkZWZxYjkEYTIt09iKKus59mPC3nl8/3UN3vIuqgfi6YNYXT6WcxqbIw1LuQfOJGOlpmUz5QGiR8NEqU6R93mzZQ/9RS16z4mJDYWx513kDhvHnZH50zlfq6obKjkwwMfklecx8aDG3F73fSN6ktWRhaz0mcxqu8obCEnn8jgqm1i6SdFLP2kiKr6ZsYPTuTH04Yy+bzkoJ2WrUHiR4NEqc5Vv2MnFU8/zbG8PCQqCsett5L0wwU9dxr7AKpuqmbtgbWsKV7DhrINNHoaSYpIYmb6TGZlzGJM/zGEhpwYw6htdPPK5/t59uNCDlU3MHxgHIumDiV7eH9sIV0bKBokfjRIlAqMxr17KX9mCdWrViF2Owk33kjS3XcRmnL6iwF7o7rmOtaVriOvKI+PSz+m3l1PfHg8M9JmMCtjFuMHjCfMFgZAo9vDW1tKeWrtPgrLaxmUHM19UwZz3eiBhNu75rRsDRI/GiRKBVZTcTHlS5ZQ9fZKMIb4719L8j33EJaREezSuq0GdwMbyjaQV5zH2gNrqWmuISY0hmlp05iVMYuJKROJsEfg8Rpydx7iiY/y2VFaTb+4cO6ZPJjbxqUTHW4//Rt1gAaJHw0SpbpGc1kZFc89T+Xrr2PcbuKuuork++4l/Lzzgl1at9bkaeLTg5+SV5zHhwc+pKqxqmX6+1kZs5gycAqR9kjW55fzxIcFbNxXQXxkKPMnZLJgQiaJ0WEBqUuDxI8GiVJdq/nIEZxLl+FasQJTV0ds1iyS7ltI5PAeOuNwF2r2Np80U7GzwUm4LZyJKROZlTGLaWnTyD/k5smPCli96zCRoTZuHZfGPZMHk5IQ2am1aJD40SBRKjjcLheu5ctxLn8R77FjRE+ZTPLChUSNHh3s0nqE4/dUWVO8hjX713Ck7gj2EDtXDLiCrIwsMiLG8uLGclZuLUMEvn/pQO6bOoShfTt2m+HjNEj8aJAoFVyeY8dwvfwKzqVL8bhcRI0bR/KihUSNH98zZxwOguN3fzw+qWRZbRk2sTG2/1jG9pnKvuJBvPVlNY1uL3Mu6s+iaUO4JC2hQ++pQeJHg0Sp7sFbV4frtddwPvc87qNHibhkJMkLFxIzbZoGyhkwxrDLuaslVIqrixGEkcmjiGi6lM92DORYbTQThybx2C2jzvre8hokfjRIlOpevI2NVL35pjXjcFkZ4cOGkbxwIbGzs3rmFPZBZIxhb+XellA5PlNx//AL8NaM4O93/oz4iLObgUCDxI8GiVLdk2lupurdVVQ8/TRNRUWEDR5M8n33Enf11Yg9sKe2nqsKqwpPuqfK2lvWEmk/u0F4DRI/GiRKdW/G4+FYbi7lTz1N4zffEJqWRtLddxN/3fcJCQvMqa29gavBdca3DfbX3iDRPqRSKujEZiPuqqsY9NabpD6xGFtCAod++1sKsmbjfGE53vr6YJfYI3UkRM5EQINERLJF5GsRyReRX7Xx+gIROSoiW30/d/u99kcR2Skiu0Xkr+IbiRORHBHZ5nvtKRHpgbdwU0q1RUJCiJ0xg8zXXiXt2WcJS0vj8EMPkT9zFuVLluCpqQl2iaoNAQsS3xf8YuBK4CLgNhG5qI1VXzXGXOr7eda37QRgIjASGA6MBab61r/ZGHOJ7/k+wE2BaoNSKjhEhJhJE8l4cTkZy18g4sILOfrIX8ifOYuj/+9xPJWVwS5R+Qlkj2QckG+M2WeMaQJWANe2c1sDRABhQDgQChwGMMZU+9ax+14/9wd5lOrFosaOJf25Z8l8/TWixoyhfPFi8mfM5Mgjj+AuLw92eYrABslA4IDfconvudZuEJGvROQNEUkDMMZsBD4EDvp+co0xu49vICK5wBHgGPBGW28uIveKyBci8sXRo0c7pUFKqeCJHDGCtMWPM+jtt4iZNpWKZ58jf+YsDv3hIZoPHQp2eb1aIIOkrauLWvce3gEyjTEjgTXAMgARGQpcCKRihc8MEZnSshNj5gADsHorM9p6c2PMM8aYMcaYMX30HglKnTMiLriAgX/5C4NXrSLuqqtwvfwy+VmzOfhvv6XpwIHT70B1ukAGSQmQ5recCpT5r2CMqTDGNPoWlwCX+R5fB3xqjKkxxtQA/wDGt9q2AVhJ+w+XKaXOIeGDB5Hy8EMMyc0l4cYbqHrzTQqyr6Tsl7+ksaAg2OX1KoEMkk3AeSIySETCgFuxvvhbiMgAv8W5wPHDV/uBqSJiF5FQrIH23SISc3wbEbEDVwF7AtgGpVQ3F5Y6kAG//S1D1qwh8c47qV6dx77vXUPJg/9Mw+7dp9+B6rCABYkxxg3cD+RiBcRrxpidIvI7EZnrW+0B32m824AHgAW+598ACoDtwDZgmzHmHSAaWCkiX/mePwI8Fag2KKV6jtB+fen3618x9P01JN17L7UbNlB43fUcWLiI+q1bg13eOU2vbFdKnZM81dW4XnoJ59JleKqqiBw9mrBBmdgTE7ElOLAlJmJzJFjLDgc2RyIh0VE6eaQfnSLFjwaJUr2Xt7YW14pXqVr1Lp7yCtwuFzQ3t7muhIZaoXI8ZBy+kEl0YHM4sPsCx+ZwYE90YEtIQEJDu7hFXUeDxI8GiVLqOGMM3tpaPE4nHpcLt8uFx+nC43LhcTlbLVuve6urT7m/kLg4K3Raejm+kGkJnQQrgHyvhcTE9JheT3uDRKfXVEr1KiKCLSYGW0wMpKe3axvT3IynsvJEyFT6QsbpxOOqtEKp0kXzwYM07NqFx+nEnKLXQ2go9oSElmBp3fPxDx1bggO7IwHp5hNXapAopdRpSGgo9j59sLfzmjSr11NnBc7xnk/rXo8vgBp376HW5cJbVXXK/YXExHz7cFtbPZ/jvZ7Y2C7t9WiQKKVUJ7N6PdHYYqIhNbVd2xi3G09l5cmH2ypb9XpcLpqPHKZhzx6r19PU1PbO7PaWw22ZK14hJDq6E1vXxtsFdO9KKaXaRex27MnJ2JOTac+NcY0xmLo63K5KPC7niTEdv56Pp7ISiYoKeO0aJEop1QOJCBIdTVh0NKS2NY1h19EbWymllOoQDRKllFIdokGilFKqQzRIlFJKdYgGiVJKqQ7RIFFKKdUhGiRKKaU6RINEKaVUh/SK2X9F5ChQfJabJwPlnVhOT6Bt7h16W5t7W3uh423OMMacdoKxXhEkHSEiX7RnGuVziba5d+htbe5t7YWua7Me2lJKKdUhGiRKKaU6RIPk9J4JdgFBoG3uHXpbm3tbe6GL2qxjJEoppTpEeyRKKaU6RIPER0SyReRrEckXkV+18fpCEdkuIltFZL2IXBSMOjvT6drst96NImJEpEef8dKOz3iBiBz1fcZbReTuYNTZmdrzGYvIzSKyS0R2isjLXV1jZ2vH5/yo32f8jYhUBqPOztSONqeLyIciskVEvhKRqzq1AGNMr/8BbEABMBgIA7YBF7VaJ87v8VwgJ9h1B7rNvvVigXXAp8CYYNcd4M94AfB4sGvt4jafB2wBHL7lvsGuO9BtbrX+PwHPB7vuLvicnwEW+R5fBBR1Zg3aI7GMA/KNMfuMMU3ACuBa/xWMMdV+i9FATx9cOm2bfX4P/BFo6MriAqC97T2XtKfN9wCLjTEuAGPMkS6usbOd6ed8G/BKl1QWOO1pswHifI/jgbLOLECDxDIQOOC3XOJ77iQi8hMRKcD6Yn2gi2oLlNO2WURGAWnGmHe7srAAaddnDNzg6/q/ISJpXVNawLSnzecD54vIBhH5VESyu6y6wGjv54yIZACDgA+6oK5Aak+b/x24U0RKgPewemKdRoPEIm08960ehzFmsTFmCPBL4DcBryqwvrPNIhICPAr8vMsqCqz2fMbvAJnGmJHAGmBZwKsKrPa02Y51eGsa1v/OnxWRhADXFUjt+l32uRV4wxjjCWA9XaE9bb4NWGqMSQWuApb7fsc7hQaJpQTw/99nKt/d9VsBfD+gFQXe6docCwwHPhKRImA8sLIHD7if9jM2xlQYYxp9i0uAy7qotkBpz7/rEuBtY0yzMaYQ+BorWHqqM/ldvpWef1gL2tfmu4DXAIwxG4EIrHm4OoUGiWUTcJ6IDBKRMKx/YCv9VxAR/1+uq4G9XVhfIHxnm40xVcaYZGNMhRA4GwAAA5VJREFUpjEmE2uwfa4x5ovglNth7fmMB/gtzgV2d2F9gXDaNgNvAdMBRCQZ61DXvi6tsnO1p82IyAWAA9jYxfUFQnvavB+YCSAiF2IFydHOKsDeWTvqyYwxbhG5H8jFOgPieWPMThH5HfCFMWYlcL+IzAKaARcwP3gVd1w723zOaGd7HxCRuYAbcGKdxdVjtbPNucBsEdkFeIB/McZUBK/qjjmDf9e3ASuM7zSmnqydbf45sEREfop12GtBZ7Zdr2xXSinVIXpoSymlVIdokCillOoQDRKllFIdokGilFKqQzRIlFJKdYgGiVIdJCL/LiK/6AZ1FPmuBVGqS2mQKKWU6hANEqXaICLRIrJKRLaJyA4RucX/f/wiMkZEPvLb5BIR+UBE9orIPb51BojIOt99L3aIyGTf80+KyBe++3/8X7/3LBKRh0Rko+/10SKSKyIFIrLQt8403z7f9N1D5Km25kwSkTtF5HPfez8tIrZA/n2p3k2DRKm2ZQNlxphLjDHDgZzTrD8Sa+qcK4B/E5EU4HYg1xhzKXAJsNW37r8aY8b4tpkqIiP99nPAGHMF8DGwFLgRa56z3/mtMw7rSuURwBDgev9CfFNg3AJM9L23B7jjDNqu1BnRKVKUatt24M8i8l/Au8aYj0XammS1xdvGmHqgXkQ+xPqy3wQ8LyKhwFvGmONBcrOI3Iv1+zcA60ZDX/leOz6Fx3YgxhhzDDgmIg1+s/J+bozZByAirwCTgDf8apmJNeHkJl/NkUBPv8+I6sY0SJRqgzHmGxG5DGvK7YdFZDXWHFzHe/ERrTf59i7MOhGZgtVTWS4if8LqafwCGGuMcYnI0lb7Oj77sNfv8fHl47+v33qvVssCLDPG/Po0zVSqU+ihLaXa4Ds0VWeMeRH4MzAaKOLE1PI3tNrkWhGJEJEkrHt7bPLdOOmIMWYJ8JxvH3FALVAlIv2AK8+ivHG+mV5DsA5hrW/1+vvAjSLS19eWRF8tSgWE9kiUatsI4E8i4sWa8XkR1iGi50TkfwOftVr/c2AVkA783hhTJiLzgX8RkWagBphnjCkUkS3ATqzp2jecRW0bgf/01bgOeNP/RWPMLhH5DbDaFzbNwE+A4rN4L6VOS2f/VaoHEZFpwC+MMd8Ldi1KHaeHtpRSSnWI9kiUUkp1iPZIlFJKdYgGiVJKqQ7RIFFKKdUhGiRKKaU6RINEKaVUh2iQKKWU6pD/D03sL28HdhzWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch3_1.best_score_, gsearch3_1.best_params_))\n",
    "test_means = gsearch3_1.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch3_1.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch3_1.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch3_1.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch3_1.cv_results_).to_csv('my_preds_subsampleh_colsample_bytree_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(colsample_bytree), len(subsample))\n",
    "train_scores = np.array(train_means).reshape(len(colsample_bytree), len(subsample))\n",
    "\n",
    "for i, value in enumerate(colsample_bytree):\n",
    "    pyplot.plot(subsample, -test_scores[i], label= 'test_colsample_bytree:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'subsample' )                                                                                                      \n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'subsample_vs_colsample_bytree1.png' )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用部分训练数据样本检验一下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>3000</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>750.000000</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>5465</td>\n",
       "      <td>2732.5</td>\n",
       "      <td>1821.666667</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.0</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3275</td>\n",
       "      <td>1637.5</td>\n",
       "      <td>1637.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>3350</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>28</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 228 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.5         3   3000           1200.0      750.000000       -1.5   \n",
       "1        1.0         2   5465           2732.5     1821.666667       -1.0   \n",
       "2        1.0         1   2850           1425.0     1425.000000        0.0   \n",
       "3        1.0         1   3275           1637.5     1637.500000        0.0   \n",
       "4        1.0         4   3350           1675.0      670.000000       -3.0   \n",
       "\n",
       "   room_num  Year  Month  Day       ...        walk  walls  war  washer  \\\n",
       "0       4.5  2016      6   24       ...           0      0    0       0   \n",
       "1       3.0  2016      6   12       ...           0      0    0       0   \n",
       "2       2.0  2016      4   17       ...           0      0    0       0   \n",
       "3       2.0  2016      4   18       ...           0      0    0       0   \n",
       "4       5.0  2016      4   28       ...           0      0    1       0   \n",
       "\n",
       "   water  wheelchair  wifi  windows  work  interest_level  \n",
       "0      0           0     0        0     0               1  \n",
       "1      0           0     0        0     0               2  \n",
       "2      0           0     0        0     0               0  \n",
       "3      0           0     0        0     0               2  \n",
       "4      0           0     0        0     0               2  \n",
       "\n",
       "[5 rows x 228 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#读取数据\n",
    "data = pd.read_csv(\"RentListingInquries_FE_train.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "y1 = data['interest_level']\n",
    "y=np.array(y1)#这么做可以防止一会for循环求准确率时出现KeyError错误。\n",
    "test = data.drop([\"interest_level\"], axis=1)\n",
    "X = np.array(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50, 227)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=5, test_size=0.001)\n",
    "X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_test = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=250, \n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.8,\n",
    "        colsample_bytree=0.9,#subsample 和 colsample_bytre用优化后的\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=0.7,\n",
       "       colsample_bytree=0.9, gamma=0, learning_rate=0.1, max_delta_step=0,\n",
       "       max_depth=5, min_child_weight=1, missing=None, n_estimators=250,\n",
       "       n_jobs=1, nthread=None, objective='multi:softprob', random_state=0,\n",
       "       reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=3, silent=True,\n",
       "       subsample=0.8)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from xgboost import plot_importance\n",
    "from matplotlib import pyplot as plt\n",
    "from xgboost import plot_tree\n",
    "xgb_test.fit(X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\preprocessing\\label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n"
     ]
    }
   ],
   "source": [
    "# 对测试集进行预测\n",
    "ans = xgb_test.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 100.00 % \n"
     ]
    }
   ],
   "source": [
    "# 计算准确率\n",
    "cnt1 = 0\n",
    "cnt2 = 0\n",
    "for i in range(0,len(y_test)):\n",
    "    if ans[i] == y_test[i]:\n",
    "        cnt1 += 1\n",
    "    else:\n",
    "        cnt2 += 1\n",
    "\n",
    "print(\"Accuracy: %.2f %% \" % (100 * cnt1 / (cnt1 + cnt2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXl8FtX1/98HkB1klyUIouxIIihgiwi2bIoiSn+KWsti3aqAFhFqLUq1LkAVtIoWBZWtKKKASEUg0m8BFTQgilGEWBBkEwVCgBDO74+ZhIeY5SHPMpfkvF+veWXmzp07n2cgz83c+7nniKpiGIZhGIVRKmgBhmEYxumBdRiGYRhGWFiHYRiGYYSFdRiGYRhGWFiHYRiGYYSFdRiGYRhGWFiHYRhRQEQmi8iDQeswjFgitg7DCBIRSQPOArJCipup6vYI2uwKTFfVhMjUnZ6IyDRgm6r+OWgtRvHC3jAMF7hSVSuHbEXuLKKBiJQJ8v6RICKlg9ZgFF+swzCcRUQ6ichKEflRRNb5bw7Z5waJyEYROSAim0XkNr+8EvAuUF9EDvpbfRGZJiKPhFzfVUS2hRynicj9IrIeSBeRMv51c0Vkt4hsEZGhBWjNaT+7bREZKSK7RGSHiFwtIpeLyFci8oOI/Cnk2odE5A0R+Zf/eT4RkcSQ8y1FJNl/Dp+LyFW57vu8iCwSkXRgCHAjMNL/7Av8eqNE5Bu//S9EpF9IGwNF5P9EZLyI7PM/a++Q8zVEZKqIbPfPvxVyro+IpPjaVopI27D/gY3TDuswDCcRkQbAO8AjQA1gBDBXRGr7VXYBfYCqwCDgKRFpp6rpQG9gexHeWAYAVwDVgOPAAmAd0AD4FTBcRHqG2VZdoLx/7V+AfwI3Ae2BS4C/iEiTkPp9gdf9zzoTeEtEzhCRM3wd7wF1gLuBGSLSPOTaG4BHgSrAq8AM4En/s1/p1/nGv++ZwMPAdBGpF9JGRyAVqAU8CbwkIuKfew2oCLT2NTwFICLtgJeB24CawAvAfBEpF+YzMk4zrMMwXOAt/y/UH0P+er0JWKSqi1T1uKouAdYAlwOo6juq+o16fID3hXpJhDomqepWVc0ALgJqq+pYVT2qqpvxvvSvD7OtTOBRVc0EZuN9EU9U1QOq+jnwORD61/haVX3Dr/93vM6mk79VBh73dSwDFuJ1btm8rar/9Z/T4bzEqOrrqrrdr/Mv4GugQ0iVb1X1n6qaBbwC1APO8juV3sDtqrpPVTP95w3we+AFVf1QVbNU9RXgiK/ZKIactmO1RrHialV9P1dZI+A3InJlSNkZwHIAf8hkDNAM7w+fisBnEerYmuv+9UXkx5Cy0sB/wmxrr//lC5Dh/9wZcj4DryP42b1V9bg/XFY/+5yqHg+p+y3em0teuvNERG4G7gUa+0WV8TqxbL4Puf8h/+WiMt4bzw+qui+PZhsBvxORu0PKyoboNooZ1mEYrrIVeE1Vf5/7hD/kMRe4Ge+v60z/zSR7CCUv6186XqeSTd086oRetxXYoqpNiyK+CDTM3hGRUkACkD2U1lBESoV0GmcDX4Vcm/vznnQsIo3w3o5+BaxS1SwRSeHE8yqIrUANEammqj/mce5RVX00jHaMYoANSRmuMh24UkR6ikhpESnvTyYn4P0VWw7YDRzz3zZ6hFy7E6gpImeGlKUAl/sTuHWB4YXc/yNgvz8RXsHX0EZELoraJzyZ9iJyje/QGo43tLMa+BCvsxvpz2l0Ba7EG+bKj51A6PxIJbxOZDd4hgGgTTiiVHUHnongORGp7mvo4p/+J3C7iHQUj0oicoWIVAnzMxunGdZhGE6iqlvxJoL/hPdFtxW4DyilqgeAocAcYB/epO/8kGu/BGYBm/15kfp4E7frgDS8+Y5/FXL/LLwv5iRgC7AHmII3aRwL3gauw/s8vwWu8ecLjgJX4c0j7AGeA272P2N+vAS0yp4TUtUvgAnAKrzO5Hzgv6eg7bd4czJf4pkNhgOo6hq8eYxnfd2bgIGn0K5xmmEL9wwjYETkIeA8Vb0paC2GURD2hmEYhmGEhXUYhmEYRljYkJRhGIYRFvaGYRiGYYRFsVqHUa1aNT3vvPOClpEv6enpVKpUKWgZ+eKyPpe1gemLFJf1uawNoqNv7dq1e1S1dqEVVbXYbM2aNVOXWb58edASCsRlfS5rUzV9keKyPpe1qUZHH7BGw/iOtSEpwzAMIyyswzAMw3CMwYMHU6dOHdq0ObEg/6GHHqJBgwYkJSWRlJTEokWLANi4cWNOWWJiIvPmzYuZrkA6DBEZKl4uAxWR9f62MlcOgDQR+cyPtb8mCJ2GYRhBMHDgQBYvXvyz8nvuuYeUlBRSUlK4/PLLATjnnHNYs2YNKSkpLF68mNtuu41jx47FRFdQk9534oU6qAdsVNV9fjygF/Hi8mfTTVX3BCHQMAwjKLp06UJaWlpYdcuXL0+ZMt5X+eHDhzmRxiT6xL3DEJHJeIHR5gMvq+pK/9RqvAidRSYjM4vGo96JUGHs+OP5xxho+oqEy9rA9EWKy/ripS3t8SsKrfPss8/y6quvcuGFFzJhwgSqV68OwIcffsjgwYP59ttvee2113I6kGgTyMI9EUkDLgx9exCREUALVb3FP96CF9BM8ZK0vJhPW7cCtwLUrl27/Zw5c2KsvugcPHiQypUrF14xIFzW57I2MH2R4rK+oLR9//33jB49mqlTpwLwww8/cOaZZyIivPzyy+zdu5f777//JH3ffvstjz/+OBMnTqRs2bJh36tbt25rVfXCQiuGY6WK9oYXMbRWyHE3YCNQM6Ssvv+zDl6U0S6FtWu22shwWZ/L2lRNX6S4rC8obVu2bNHWrVsXei63vq5du+rHH398SvfidLHV+knjpwB9VXVvdrn6eZhVdRcwj5PTSRqG4Th5OX1ef/11WrduTalSpViz5oSXJZ5On9OVHTt25OzPmzcv57nu2LEjZ5L722+/JTU1lcaNG8dEQ6ArvUXkbOBN4Leq+pWIDAXuwEtBmYmXAvIoXmrM0YEJNQzjlBk4cCB33XUXN998c05ZmzZtePPNN7nttttOqpvt9ClTpgw7duwgMTGRK6+8MmZj8a4zYMAAkpOT2bNnDwkJCTz88MMkJyeTkpKCiNC4cWNeeOEFAD777DMeeeQRzjjjDEqVKsVzzz1HrVq1CrlD0Qj6X+MvQE28bF4AzYFWeJ3DVXjJXs7ES5rzc4+ZYRjOkpfTp2XLlnnWjafT53Rg1qxZPysbMmRInnV79OjB3/72t1hLAgJah6GqjVV1j6reoqrVVTUJzyVVGs89dSPQX1UT1c+pLCJnBaHVMIz48OGHH9K6dWvOP/98Jk+eXGLfLlzGqfDm2e4p4F6gvKreKyIdgJVAR1Vdm8c1OS6pWrVqt//L0/+Mo+JT46wKsDMjaBX547I+l7WB6cvm/AYnZ7DN7fTJZvjw4dxxxx00b94cICpOn1jhsoMLoqPPaZdUfhu+ewqoCkwFUvByMX8MJBZ2vbmkIsNlfS5rUzV9+ZGf0+fSSy89yckTDadPrCgJ/7acLi6pvFDV/ao6SL2hqpuB2sCWgGUZRtx46qmnaN26NW3atGHAgAEcPnyYpUuX0q5dO5KSkujcuTObNm0KWmbUiKfTxyg6rnUYVYCVIjJHRMqKyEVAFpCmqvsD1mYYceG7775j0qRJrFmzhg0bNpCVlcXs2bO54447mDFjBikpKdxwww088sgjQUstkAEDBnDxxReTmppKQkICL730EvPmzSMhIYFVq1ZxxRVX0LNnT8Bz+iQmJpKUlES/fv1i6vQxio5rs0pVge5AOeBzoD6wC294yjBKDMeOHSMjI4MzzjiDQ4cOUb9+fUSE/fu9v5t++ukn6tevH7DKgsnL6QPQr1+/n5XF0+ljFB1nOgw/xpQCrwAvA//AW4txEZAeoDTDiCsNGjRgxIgRnH322VSoUIEePXrQo0cPpkyZwuWXX06FChWoWrUqq1evDlqqUcJwpsNQ1dtFpBdemJBywEzgMrwOIyws+GBkuKzPZW0Qub7QwHP79u3j7bffZsuWLVSrVo3f/OY3TJ8+nTfffJNFixbRsWNHxo0bx7333suUKVOiId8wwsJVW+3zwARVXS0i04CFqvpGPtdY8MEo4bI+l7VBdPUlJyfz0UcfMXLkSAD+/e9/88UXX7BmzRpmzJgBwM6dO7n//vuZNm1a3PXFApf1uawNzFZbC88RleZvB/HmMa4u7Hqz1UaGy/pc1qYaXX2rV6/WVq1aaXp6uh4/flxvvvlmnTRpktasWVNTU1NVVXXKlCl6zTXXBKIvFrisz2VtqvG11TozJBWKqp6TvR/yhvFWcIoMI39SU1O55ZZbcv7K27x5M2PHjqVr167cfvvtHD58mDJlyvDcc8/RoUPhMTQ7duxI//79adeuHWXKlOGCCy7g1ltvJSEhgWuvvZZSpUpRvXp1Xn755Vh/NMM4iUA6jJAggy2Az/zig8AZ/vnywAq8uYyzgYpAnkNShhE0zZs3Z8qUKXTt2pWsrCwaNGhAv379+P3vf8+YMWPo3bs3ixYtYuTIkSQnJ4fV5sMPP8zDDz98Ulm/fv3ydBgZRrwIah3GncDlwC+BS1W1LfBXYJt6SZWOAJepaiJePoxGItIpIK2GETZLly7l3HPPpVGjRqedDdYwCsPJFK3+mNpBv/wMfyt0dt5cUpHhsj7XtOWXTnP27NkMGDAAgKeffpqePXsyYsQIjh8/zsqVK/O8xjBOF1xO0VoaWAucB/xDVe/Ppy0LPhglXNbnmrbcQfYOHjxIuXLl6N+/P1OnTqVGjRpMmjSJxMRELr30UpYvX87ChQuZMGFCIHpLgtMnVrisDUqAS4owUrSGnKsGLAfaFNauuaQiw2V9LmtT9fS99dZb2r1795yyqlWr6vHjx1VV9fjx41qlSpWg5J0Wz89VXNamWsJcUiEpWntrSIrWbFT1RxFJBnoBG+IszzAKJdsltXfvXqpUqULVqlUZO3YsNWvWpHXr1pQtWzYnvIdhnM4EGnwwrxStIrJRRFRENojIehFZDfQFvgxSq2HkR/PmzXn22WcpVaoUKSkpVKxYkX79+lGzZk0yMzNRVUSESpUqBS3VMCIi6Gi1oSlaU4An8NxTNwPH/Tp1gTqqujAYiYZROOXLl2fv3r2sWbMmxyVVo0YNHnnkEdatW8dDDz2UkyzIME5XAhmSUtXG/u4t/pbtnmrFCfdUW7+8OjYUZZwmmEvKKM44GUtKC3BP5XGNuaSihMv6XNNmLqno4rI+l7VBCXBJ5bdxCu6pvDZzSUWGy/pc1qZqLqlIcVmfy9pULUUrcJJ7qq/m4Z4yDNeYNWtWznAUQP369fnggw8AWLZsGU2bNg1KmmFEhcBttXmR2z0VtB7DCCU1NZXrrrsu53jz5s0MGDCAJUuW0K5dO5o3b54TNPCPf/wjx44do3z58rz44osBqjaMyHEy+CAnu6eaAUdVtVrchRpGHjRv3pyUlBSAnGCD3bp14/rrr+fRRx9l/fr1lCtXjl27dlGnTp2A1RpG9AjqDeNOoDdQD9ioqvtEpDfwkHoT3rcAt4jIvXgJlaoGpNMwCiQ72GDdunV57rnnGDVqFOXKlQOwzsIodjgZfNCvlwBcATwK3BtO2xZ8MDJc1he0tnCCDX711Vf85z//4YEHHqB8+fKMHz+eiy4KO8OwYTiPy8EH3wAeA6oAI1S1Tz5tWYrWKOGyPhe1ZWZm5thoy5Yty913380FF1zA3XffzZdffsnYsWOZOXMmIhK0VCefXygu63NZG5QAWy2F2GeBPsBz/n5XvIx7ZquNMS7rc1FbqI12+fLl2rNnz5N0NmnSRHft2hWQupNx8fmF4rI+l7WpljBbbT722V8CV/lvIrOBy0RkekASjWLIjz/+SP/+/WnRogUtW7Zk1apVpKSk0KlTJ5KSkrjwwgv56KOPCmwjt4326quvZtmyZYA3PHX06FFq1aoV089hGPEkqA6jCrBSRBYBHwINgatCzj8HfA1k+NtmVb0p7iqNYsuwYcPo1asXX375JevWraNly5aMHDmSMWPGkJKSwtixYxk5cmS+1x86dIglS5ZwzTXX5JQNHjyYzZs306ZNG66//npeeeUVJ4ajDCNaBOWSqgp0x5vMzgR+Au4RkevVG0c7BvxRVT/x3VOvi0grVf0iIL1GMWL//v2sWLGCadOmAVC2bFnKli17SilVK1asyN69J68nLVu2LNOn24uwUXwJyiWlwCt4LqmbROQh4KCqjgdQ1R3ADn//XRFZCjQACuwwzCUVGS7ri1RbqMtp8+bN1K5dm0GDBrFu3Trat2/PxIkTLVigYRSCEy6p3B1GrrqNgRV4Gff253Hegg9GCZf1RaotNFhgamoqd955J8888wytWrXimWeeoVKlShw8eLDIwQJLgpMmlrisz2VtUDJdUg/hWWdz16uMl9f7mnDaNZdUZLisL5raduzYoY0aNco5XrFihV5++eURBQt0+dmpmr5IcFmbaglL0ZofInIGMBeYoapvBq3HKD7UrVuXevXq0bNnT7799lv27NlDz549LaWqYRRC4LbaUEJStM4FtgO/wjGNRvGgVq1apKamUrZsWX7xi1/w6KOPWkpVwyiEQN8wRKQusAbPNXUcqAS0AxoB1wC78NxTNwF/UtVFQWk1ig/79+9nw4YNbNmy5STba40aNRgxYgTXXXcds2bNYsGCBQGqNAz3CDpFK/jxo3z31GBgJp57SgqaDDeMomIuKcMoGs7MYajq7SLSC+imITGmTgWz1UaGy/qiaas9duwYn3zyCc888wwdO3Zk2LBhPP744/z000889dRTXHvttcyZM4chQ4bw/vvvR0O+YRQLnM7pHc4bhgUfjB4u64umth9++IE777yT2bNnA7B+/XpmzpzJhg0bWLBgASKCqtKnTx/eeSe8TsrlZwemLxJc1gYlwFab30aYdtv8NrPVRobL+qKtrXPnzvrll1+qquqYMWN0xIgR2qJFi5z7vP/++9quXbvA9EUb01d0XNamarZaowTSuHFjqlSpQunSpSlTpgxr1qzJOTd+/Hjuu+8+du/eHbVgfs888ww33ngjR48epUmTJkydOpW+ffsybNgwS6lqGPngZIpW3z21FqgDlBKRx4Beqro0ALlGnFi+fPnPOoStW7eyZMkSzjrrrKjeKykp6aROCaBz586sXbs2qvcxjOJEUGsc7gQuxwtjfqmqtgX+CmxT1T2q+j3wPnCHqpbGs9vab3IJ5J577uHJJ58MWoZhGDiaolVEqgJdgIEAqnoUOFpY2+aSiox46sud8lRE6NGjByLCbbfdxq233sr8+fNp0KABiYmJcdFkGEbBOBF80C/LSdEqIknAi3jRaRPx3i6GqWp6Hm1Z8MEoEU99ocEAAfbs2UOtWrXYt28fI0aMYOjQoUyePJlx48ZRuXJlrrvuOl588UXOPPPMfFoMlpLgpIklLutzWRuUAJcUhadozc6J0dE/ngj8tbB2zSUVGa7oGzNmjI4dO1Zr166tjRo10kaNGmmpUqW0YcOGumPHjqDl5Ykrzy4/TF/RcVmbaglzSYWkaO2tJ1K0bsObz/jQP34DGBWEPiM2hLqiRIQVK1YwduxY3n77bb777jsSExP56quvqFatGuAFDPzkk08s5alhBEhQHUZ2itatQEfge2C6iAxW1Q2q+r2IbBeRxUBjoCbwXkBajRiR7YravHkznTt35sCBA1SoUIE//elP7N+/n8cee4wnnngiaJmGYfgEnaJ1GlAaz1JbBVjpnwPYA7QF9vvlf4q7SiMuNGnShHXr1p1UNm/ePN54442c49mzZ9vbhWEETNxttblStJ4LdFfVJFVtCewWkbN8l9T5wDmq2lZV+6rqt/HWasSObFdU+/bt81wg9/LLL9O7d+8AlBmGkR+BuqSAe4HyqnqviHTAe5PoCGRhLqm4E0t94biisu2z06dPJzU1lbFjx+aEHy8JTpVYYvqKjsvaoAS5pPCGn6YCKcBrwMd4HYS5pAIgKH1jxozRcePGqarqtGnTtFOnTpqenn5SHXt2kWH6io7L2lTj65IKNJudqu5X1UGqmgTcDNQGtpC3S6pdQDKNKJOens6BAwdy9t977z3atGnD4sWLeeKJJ5g/fz4VK1YMWKVhGLkJOuNeNeCQeiu5bwFWqOp+YL+IbBWR5qqaipeq9YsgtRqREWqjzcrKolSpUmRlZZGWlka5cuWYMGEC33zzDZmZmXTv3h2ATp06MXny5ICVG4aRTdDrMFoCr4pIFl6H8KGIbATqAruBT/0x7DVA38BUGlEhd3DBkSNHUqNGDUaNGsXjjz9Ou3btzEZrGA4TdIrWPUDT7HIR+RLojddZpKuq+gv75qjqvrgLNWLK22+/TXJyMgC/+93v6Nq1q3UYhuEwQb9h5JBHUMKn/FOV8Gy4hWLBByMjmvrCCS64c+dO6tWrB0C9evXYtWtXVO5tGEZscDZFq4j0Ax7Dy4lxhaquyucaS9EaJWKpLy8b7QMPPMDChQtz6lx55ZUsWLAg7tqigemLDJf1uawNSoCtNr+NXEEJ/bIuwPvhXG+22siIl75sG22zZs10+/btqqq6fft2Lejfz55dZJi+ouOyNtUSZKsNB1VdAZwrIhYXwnGysrK44IIL6NOnD+D9MfLAAw9w3nnn0axZMyZNmnSSjfaqq67ilVdeAeCVV16hb1/zNRiGywTSYYjIUBHZKCIqIuv9bSUnUrR2FpHlfp1vgOrA3gIbNQJn4sSJtGzZMud42rRpbN26lcWLF1OhQgVeeOEFOnTowBVXXEGvXr0YNWoUS5YsoWnTpixZsoRRoywgsWG4TFCT3nfiuaHqARtVdZ+I9Abe8s/3As4GDuAFHzwDz4JrazEcZdu2bbzzzjs88MAD/P3vfwfg+eefZ+bMmZx33nk/Cy4IULNmTZYutTTthnG64FqK1j3qZeH7s79lX/M20IBCOgxzSUXGqejL7YIaPnw4Tz75ZM4KboBvvvmGf/3rX8ybN4/atWszadIkmjZtmrspwzBOE+LeYajq7SLSC+imISlagSHAu7nri0hj4ALgw9zn/POhwQf5y/nHoi05apxVwftSdpVT0Ze9fgJg1apVZGZmcuDAAVJSUti7dy/JyckcOnSI7777jvHjx7NixQquvfZaJk2aVCRtBw8ePOmermH6IsNlfS5rgzjrC2dmPNobhaRoDSmvjBep9ppw2jWXVGQUVd+oUaO0QYMG2qhRIz3rrLO0QoUKeuONN2rz5s11y5Ytqqp6/PhxrVq1aty1xQvTFxku63NZm2oJc0mFpGjtqydStCIiZwBzgRmq+mZQ+k5nDh8+TIcOHUhMTKR169aMGTMGgCFDhpCYmEjbtm3p378/Bw8ejOg+jz32GNu2bSMtLY3Zs2dz2WWXMX36dK6++mqWLVsGwAcffECzZs0i/kyGYQRHUB1GdorWRXhDTQ2Bq7JPihdA6l3gIuBOETH7TBEoV64cy5YtY926daSkpLB48WJWr17NU089xbp161i/fj1nn302zz77bEzuP2rUKObOncv555/P6NGjmTJlSkzuYxhGfAg6Reu9QCbwE3CPiFyv3mrDS/Ai1KYCR4AxIrJPVV8ISO9piYjkrADNzMwkMzMTEaFqVS8LrqqSkZGRk6QoGnTt2pWuXbsCUK1aNd55x91JfsMwTo2gU7SuVdWqwGTgKT2xND0TeE9VW6hqIjAWqBFvrcWBrKwskpKSqFOnDt27d6djx44ADBo0iLp16/Lll19y9913B6zSMIzTgUBTtKrvkhKRh4CDqjreP+4P9FLVW/zj3+Jl37srj7YsRWsucqdDBc9J8eCDDzJ06FDOOeccwOtMJk2aRIsWLejdu7fTMXNc1gamL1Jc1ueyNohvLClnotXmIq8xkjx7NlV9ES//N82bN9e7b3Q3vERycjL/zx+uCYK1a9eyd+9eBg0alFNWpkwZxo0bxxNPPEFycnLOcJJruKwNTF+kuKzPZW0QX32Bu6TyYRveRHg2CcD2gLSctuzevZsff/wRgIyMDN5//32aN2/Opk2bAG8OY8GCBbRo0SJImYZhnCa4+obxMdBURM4BvgOuB24IVpL7HD58mC5dunDkyBGOHTvGJZdcwocffsiuXbvYvXs3mZmZzJ07l379+rF//35UlcTERJ5//vmgpRuGcRoQVIeRbav9Cs8tVRY4LCLDgVaqul9ENgFfA1nAWFX9PCCtpw3ZNtrKlSuTmZlJ586def755ylXrhzVq1ena9eulCpViv/+979BSzUM4zQkqA5jN17wwXSgEXA1sC970tvnEWAU8KqqPhp/iacf+dloL7jggoCVGYZRHHAh+OBTInJF7nqqusKPIxU2JTX4YGggwKysLNq3b8+mTZv4wx/+kGOjNQzDiJRTttWKSHWgoaquL/JNC7HVhtRrDCxU1TYFtFXibbXh2mivv/56XnjhBc488+f1s69x1T7osjYwfZHisj6XtYGDKVqBZLzV2TWA/+EFBPx7ONfm014aJwcffAgYkUe9xsCGcNu14IMn89BDD+m4ceNyjhs1aqS7d+/Ot77LQdZc1qZq+iLFZX0ua1N1M/jgmaq6H7gGmKqq7YFfh3mtcYps3bqVbt260bJlS1q3bs3EiRNPOj9+/HhEhD179pxUnpeN1iyzhmFEi3A7jDIiUg/4f8DCWIkJSd06Q0S6AouA80Tkg1jd00XKlCnDhAkT2LhxI6tXr+Yf//gHX3zh5Y7aunUrS5Ys4eyzz/7ZdTt27KBbt260bduWiy66iO7du9OnTx8mTZpEQkIC27Zto23bttxyyy3x/kiGYRQDwp30Hgv8G/ivqn4sIk3wLK8RISJ1gTV4w13HgUpAO2Ar8C1wGCiNtyZjiKq+FOk9Twfq1atHvXr1AKhSpQotW7bku+++o1WrVtxzzz08+eST9O378xXtbdu25dNPP/1Z+dChQxk6dGjMdRuGUbwJq8NQ1deB10OONwPXFvWmqto45DABctxTg4GZwGzgGVX988+vLlmkpaXx6aef0rFjR+bPn0+DBg1ITEwMWpZhGCWQsFxSItIMeB5H9vBSAAAgAElEQVQ4S1Xb+EmPrlLVR6IqxndP4eXzPgNojbfIb6KqvprPNcXGJZXb7ZSRkcGwYcO46aab6NChA/fccw/jxo2jcuXKhTqeioLLbhCXtYHpixSX9bmsDdx0SX0AdAA+DSkL270U7obvngKeBVbjDVHVwhv+albY9cXJJXX06FHt0aOHTpgwQVVV169fr7Vr19ZGjRppo0aNtHTp0tqwYUPdsWNHIPrijcvaVE1fpLisz2VtqvF1SYU7h1FRVT/KlWjnWJjXFoVtwB5VTQfSRWQFkAh8FcN7OoOqMmTIEFq2bMm9994LwPnnn8+uXbty6jRu3Jg1a9ZQq1atoGQahlHCCNcltUdEzsUPMe7nq9gRM1XwNnCJiJQRkYpAR2BjDO8XKLlttMOHD+e1115jxowZlC9fngoVKnDxxRfnWGYNwzCCINwO4w/AC0ALEfkOGA7cHgM9VYCVwN+AesBB4AdgvapuiMH9nCC3jfbdd9/l888/Z8aMGRw8eJCMjAy6dOnCY489lnNNWlqavV0YhhFXCu0wRKQUXhiPXwO1gRaq2llVv42Bnt1AT7zw5gtUtTxeXoxeIlI2Bvdzgnr16tGuXTvgZBttjx49KFPGGzXs1KkT27ZtC1KmYRglnELnMFT1uIjcBczx5xRiQq6ghDOBKuJNmlTGe8sodM7kdAs+GBo0MKcsxEYbyssvv8x1110Xc42GYRj5Ea6t9kEgA/gXXkhyAFT1h6iKOWGrPYLXcbTAG6a6TlXz7AlCbbW1a9duP2fOnGhKiiqF2d9CbbRdunTJKZ8+fTqpqamMHTuWXMaDuOoLEpe1gemLFJf1uawN3LTVbslj2xzOtaeyccJW2x94Ci+393n+/aoWdv3pbKvNbaPNZtq0adqpUydNT0+PsTq37YMua1M1fZHisj6Xtak6GHxQVc/JY2tyip3YqTAIeNP/LJv8DuO0jqI3ePBg+vXrR5s2JyK1p6Sk0KlTJxITE6lbty7Vq1fPsdECLF68mCeeeIL58+dTsWLFIGQbhmHkENY6DBG5Oa9yzWf1dch1Q4E7gC+A+nhxoh7QkLwXItILmIgXMyp72fL/gNEiMhU4F/ge2ByOVlcZOHAgnTp1YtKkSTllI0eOZMyYMVSpUoVLLrmEhQsXkpSUBMDf/vY3hg4dypEjR+jevTvgTXxPnjw5EP2GYRjhLty7KGS/PPAr4BOgwA4DuJOfp2LNQURKA//Ay+u9Dc9G2wz4KzAXb6I7E3hY/WRLpytdunRh+/btJ5WJCPv376d3797MnDmTBQsWMHPmzJzzl19+ebxlGoZh5Eu4wQfvDj0WkTOB1wq6JsxUrB2ATeoFM0RExgCXqupK4GK/LA14Ixydrrmk8nJBhfL000/Ts2dPRowYwfHjx1m5cmWclBmGYZw6Rc3pfQhoWlAFVb3dH27qVsDbQQO8UObZbMNb1R02uYIP8pfzYxmx5NRITk4+6Tg9PZ309PSc8kmTJjFkyBAuvfRSli9fzjXXXMOECRPiL9Tn4MGDP9PsCi5rA9MXKS7rc1kbxFlfODPjwAK8N4X5eAmUNgNPhHFdGgWkYgV+A0wJOf4tXljzfNsoaHPdJTVr1ixt3bp1znHVqlX1+PHjqqp6/PhxrVKlSlDSVNVtN4jL2lRNX6S4rM9lbaoOuqSA8cAEf3sM6KKq90fYV4H3RtEw5DgB2J5PXScZPHgwderUOcn99OCDD9K2bVuSkpLo0aPHz+Yusqlfvz4ffOAlE1y2bBlNmxb40mYYhhEo4Q5JXZ67gxCRJ8LoNKoAK0XkMzyX1EXAu3gdEHghQC4WkT3ATrxJ7hv89v8K9PWve11EblRV5zqTgQMHctddd3HzzSeMZPfddx9//etfAW/YaezYsfz000+899577N+/n4SEBB5++GH++c9/MmzYMI4dO0b58uV58cUXg/oYhmEYhRJuh9EdyN059M6jLDdV/WuPAMuALODXIrINaKWq+0XkEbzghucBY1X1c//a/UANvHhX7YClQMsw9caNLl26kJaWdlJZ1apVc/bT09MREWbNmkVycjJdu3Y9qe7atWvjoNIwDCNyCuwwROQOPGtsExFZH3KqCvDfQq6djBcO/RU8l9RZIvIQcFBD1mGo6pMiMgdYqKqPhpSPA8b5bY0Gzj6VDxY0DzzwAK+++ipnnnkmy5cvD1qOYRhGxBQYS8q3z1bHm7cYFXLqgIYRRyo7NpT6Lqm8Ogy/vDFeh9EmV/mjwM3AT3huq9153CPuKVpzp1L9/vvvGT16NFOnTv1Z3RkzZnD06FEGDRpUImLSxAqXtYHpixSX9bmsDRyMJaUnHEt18P7SPxs4O4z6aRTgkgopb0wBKV+B0XiL95x0SW3ZsuUk91MoaWlpOedKgtsiVrisTdX0RYrL+lzWpuqgS0pErhSRr/FiOn3gdwTvhtl5RYOZwLVxvF9EfP311zn78+fPp0WL0zoMlmEYBhD+pPcjQCfgfVW9QES6AQNiJwtEpKmqZn/zXgV8Gcv7nSqDBw9m4cKFHDlyhIoVK7Jnzx6qVKlCpUqVyMjIQFVp2LAhTZo0sfhPhmEUC8Jdh5GpqnuBUiJSSlWXA0nh3kRE6vrOqHuBP4vIjyKSKiJzfUvtFqCliGwTkSEiUh74UEQyROQwcBcw7NQ+WmwZOHAgixcvpmHDhuzYsYPMzEzmzp3Ltm3b+Omnn7jjjjvo06cPCxYsoEGDBkHLNQzDiJhw3zB+FJHKwH+AGSKyizAy4Klq45DDhOwdEfmSnwcl3Kf+ZLifae9sVT0oImcA/4e3wO+7MPXGnLzstD169MjZ79SpE2+8EVYILMMwjNOCcDuMvngZ94YDN+KFIR9blBuGE5TQn4Q56B+e4W+FpgaMdfDBwoIJhmIpVQ3DKG6ElaIVQEQaAU1V9X0RqQiUVtUDRbppGHZbP/T5WrwFff/QfFaVx9NWG66dNr+UqiXBnhcrXNYGpi9SXNbnsjZw0FYL/B4vjMc3/nFTYGk41+bTXhph2G39c9WA5UCbwtqNt602LzttQSlVS4I9L1a4rE3V9EWKy/pc1qbqoK0WL3THL/HCdaCee6lOmNdGhKr+CCQDveJxv8LIK9jg66+/TuvWrRERHn74YUupahhGsSTcDuOIqh7NPhCRMoQxp1AA2UEJ9/khR24HholIZ7/92iKSJSIpIrIOGIojttpsd9TWrVu5+OKLSU1N5e677+aGG26gfPnypKen0717d5KSkrj99tuDlmsYhhE1wp30/kBE/gRUEJHuePGlFkRw3+yghD8CK4DsP8eXi0hNoJ5/XMrfJqrqwgjuFzWy3VENGzZkw4YNJ51bsmQJ48eP58ILCx8KNAzDON0It8MYBQwBPgNuAxYBU4pywzyCEib45Rf7x/uB9SKSoapti3IPwzAMI/oUFnzwbFX9X9RvGuKSEpF+eMEN6wBXqOoqv84xIAVvvcfjqvpWPm3F3CUVrjtq+PDh3HHHHTRv3jzPdkqC2yJWuKwNTF+kuKzPZW3gkEsK+CRkf244s+jhbOSRdhXoghd6JPu4vv+ziV//3MLajZdLKr9gg5deeql+/PHH+V5XEtwWscJlbaqmL1Jc1ueyNlW3XFISst8k/P7q1FHVFcC5IlLLP97u/9yM55K6IJb3NwzDMAqmsA5D89mPCiJynh8GBBFpB5QF9opIdREp55fXwrP0fhHt++dFamoqSUlJOVvVqlV5+umnc84PGDAgxx2VkJDASy+9xLx580hISGDVqlVcccUV9OzZMx5SDcMw4kphk96JIrIf702jgr+Pf6yqWjX/S8PiWuBmEcnECz3yOl7HUAuoISLZVt5/q2pcOozmzZuTkpICQFZWFg0aNKBfv34552fNmpXndaF1DMMwiiMFdhiqWjoWN9UTQQmf8DfgpKCEjfBWfveJxf3DZenSpZx77rk0atQoSBmGYRhOEK6tNubkDkpYlDaKGnwwv6CCs2fPZsCAmKb9MAzDOG0IO/hgPMi22wJtgLnANmA73tvG5/lck2OrrV27dvs5c+ZERUtmZib9+/dn6tSp1KhRIyptlgR7XqxwWRuYvkhxWZ/L2sAhW228N3y7Ld5K8Mp+2eXA1+FcH01b7VtvvaXdu3ePWnuqJcOeFytc1qZq+iLFZX0ua1ONr63WmSGpUNRb7Z29v0hEnhORWuqHQ482jRs3pkqVKpQuXZoyZcqwZs0aZs2aZcNRhmEYIQTSYYjIUOAOvICCZYCz/Z+V/fN1gZ14QQo3ARWAvbHUtHz5cmrVqgXAoUOHWLJkCS+88EIsb2kYhnFaEW602mhzJ95Q08fAF6qaCHQFquNl1+sPbAC+8TX+239tigsVK1Zk7969nHnmmYVXNgzDKCHE/Q0jlxtqJlDFX7xXGdgC7FTVZ0VkFXAfsBhvIrxQwnVJ5XZFiQg9evRARLjtttu49dZbT+UjGYZhlAgCcUmFuKGO4HUcLfCGn65T1XdEpBSwDPgt8Cu8QIV35dPWKQcfzB1McM+ePdSqVYt9+/YxYsQIhg4dSmJiYlE/Xr6UBLdFrHBZG5i+SHFZn8vaIL4uqaAnvXviRaS9DDgXWCIi/wFuBhap6tbQnNh5oaovAi8CNG/eXO++sW9EgtatW0dmZiZdu3aNqJ28SE5Ojkm70cJlfS5rA9MXKS7rc1kbxFdf0B3GILzQ5QpsEpEteG8bFwOXiMideENVZUXkoKqOiraA9PR0WrZsmTNfsWXLFt54441o38YwDOO0J6hJ7yrASrzOYaaIHBGRh4DmwGZVvRGYABzEG7ZaF4vOAmDnzp18//33qCrHjx9n9OjR9OrlRPpwwzAMpwjqDSM7RWsm8E+gHF4mv/vUS6rUBvg90AG4HnhYRJqq6tfRFtKkSRPq169PcnJyjq3WMAzD+Dlxf8PIlaL116raCXgOmKCq0/1qLYHVqnpIVV/Gm6OIWTjYbJdU+/btefHFF2N1G8MwjNOaQF1S2Su3/eGog6o63j9uCbyNN5eRASzFW7p+dx5tmUsqSrisz2VtYPoixWV9LmuDkuWSyhNV3SgiTwBL8OYx1uHl9s6rrrmkooTL+lzWBqYvUlzW57I2iK++oCa9C0VVX1LVdqraBfgBiPr8BXguqQMHDuTsv/fee7Rp0yYWtzIMwzitcfINA0BE6qjqLhE5G7gGb3gqqmRlZXHRRRexbds2zjnnHI4dO8YNN9xgLinDMIw8CKrDqAKsFJGmeE6p0oCKyH1AUz9a7TL/PMCrqrov2iImTpxIUlISTZo0YeHChdFu3jAMo1gR1JDUbrxV3r8EzlIvFeyVQJqq7heR0ngRalvidS4dRKRVNAVs27aNd955h1tuuSWazRqGYRRbgg4++LKqrvRPrQYS/P0OwCZV3exfMxvoC3xRUNuFBR8MDTo4fPhwnnzyyZz5C8MwDKNg4t5hqOrtItIL6JYrIdIQ4F1/vwGwNeTcNqBjXu3lStHKnF6V8r13cnIyAKtWrSIzM5MDBw6QkpLC3r17c87FkoMHD8blPkXFZX0uawPTFyku63NZG8RZXzhp+aK94adiDTnuBmwEavrHvwGmhJz/LfBMYe2Gm6J11KhR2qBBA23UqJGeddZZWqFCBb3xxhvDujYSSkKqx1jhsjZV0xcpLutzWZtqCUvRKiJtgSlAb1XNzqq3DWgYUi0B2B6N+x0+fJilS5dSs2ZNjh07xqWXXsrevXuZPn164RcbhmGUYIJ2SW3Ce7soDVwFjPfPfww0FZFzgbl4cx5RsdWWK1eOZcuWUblyZTIzM2nbti01atSIRtOGYRjFmqBcUlXxggr+gGer/QG4R0TWAKjqMeAuYBVenoztqvp5NG4sIjnL6DMzM6lQoQJ///vfo9G0YRhGsSbo4INrVbUqMBl4Sk+OZbIe+AzPHbUpmhqysrJISkqiTp06dO/enY4d85xPNwzDMEJwMvigX/YG8Bje8NUIVe2TT1uFBh/MHWwwm4MHD/Lggw8ydOhQzjnnnEg+UliUhCBmscJlbWD6IsVlfS5rAws+iIj0AXap6loR6VpQXY0w+ODatWvZu3cvgwYNKqrcsLEgZkXHZW1g+iLFZX0ua4OSlaI1P34JXCUilwPlgaoiMl1Vb4q04a1bt9K3b1+ysrI4evQo6enpPPfccxELNgzDKO4E7ZKqhJezOwPIEpGKqjoWmAb09utWBmoCa6Jx4x9++IGsrCxUlVKlSnH8+HHLtGcYhhEGQadobQzMAAQ4DtwqIk+raiqQBCAilwGLgHnRuHFiYiLr1q0D4NChQ3Tu3BkRiUbThmEYxZqgXVKNgKWqWlVVq6lqgnqRakMpg+em+jZaGswlZRiGceoEGksKaAP8WUTW4a3kHpHHeovrgVnhtJ1f8MHQoIMApUuXJiUlhR9//JF+/fqxYcMGS5pkGIZRCIHaaoGjwHFVPehPcE9U1aYh9cridSStVXVnPm2FBh9sP2fOnFPS8sorr1C+fHmuu+66In2WU6Ek2PNihcvawPRFisv6XNYG8bXVOhF8ML9yvEV774XbbjjBB3ft2qX79u1TVdVDhw5p586ddcGCBYVeFw1KQhCzWOGyNlXTFyku63NZm2p8gw8GmtNbROqKP+MsIh3w5lT2hlQZQJjDUeGSlpZGgwYNqFChAtWrV6d06dL06ZPnmkDDMAwjhEA7DKA/sMGfw5iEF2jwCxGZISLP4YU5/6OItIvWDS+88EJ27txJRkYGBw4cICMjg9WrV0erecMwjGJLILZaVW3s7z7rbwCIyJd46y9aAnfj6esIPE8+CZROldzBBzMzM81WaxiGEQbOrPTOlbq1GTDQH1tbLSLVRKSequ4oqI1wXVJZWVm0b9+eTZs28Yc//MFstYZhGGEQiEsqP0LcU9OAx1X1//zypcD9qvqz1d4WfDB6uKzPZW1g+iLFZX0uawMLPgjeyu/c5NmzqQUfjBou63NZG5i+SHFZn8vaIL76gp70zo+op2jdunUr3bp1o2nTprRo0YKJEyeSkZHB+++/T4sWLSISaxiGURIIpMMQkaEislFE5onIAhFZJyKf4wUaBG8eY6aIpIjIV0CdwuYvCqNMmTJMmDCBuXPnUq5cOUaOHEnbtm3p3r272WoNwzDCIKghqTvx3FADgDNV9UoRqQ18D5yBF2wwEy+q7SG8QIURUa9ePerVqwfAunXr6Nu3L3fddRfdu0fctGEYRokg7h1GLjfUTKCKv3ivMrAF2KmqKiJHVfXcWGhIS0vj008/NXeUYRjGKRB0LKkjeB1HC7y3ietU9R2/zjEgBTiG55h6K5+28nVJ5eWOysjIYNiwYdx000106dIlip+qcEqC2yJWuKwNTF+kuKzPZW1QgmJJ4a30fgrPFXUe3htGVb9Off9nE7/+uYW1W1gsqaNHj2qPHj10woQJYcVXiTYlISZNrHBZm6rpixSX9bmsTbUExZICBgFv+po34XUYLQBUdbv/czOQDFwQyY1UlSFDhtCyZUvuvffeyFQbhmGUQILuMP4H/ApARM4CmgObRaS6iJTzy2vh5fj+IpIbvfnmm7z22mtMnjyZ8uXL06BBAxYtWhShfMMwjJJDzDqMEOvsXBFZJSJHRGRErmrVgNEichjIXs29By+W1Ba/fBuwSVUj6jB+8YtfsHbtWg4fPszu3bupVKkSjRs3jqRJwzCMEkUsXVLZ1tl0vFSsV2efUD/4oIg8D4wDXlXV0JR35fDeKK5Q1SMiUidSMaG22ipVqtCyZUu+++47WrVqFWnThmEYJYKYdBi5rLMvq+pTInJF7nqqukJEGufRxB14zqgjfr1d4dw3d/DB3EEHc8rNVmsYhnHKxMxWm22d9YeYEJGHgIOqOj5XvcbAwtA3DBFJAd4GegGH8XJ9f5zPfU4pRavZavPHZX0uawPTFyku63NZGxQTWy0/T7f6EN4Xf+56jYENuco24CVUEqADnntKCrun2Wojw2V9LmtTNX2R4rI+l7WpxtdW62q02m34dlvgIxE5jrduY/epNjR48GAWLlxI7dq1ad++vdlqDcMwikjQwQfnAm8CrXI5qN4C3hKRz0RkI17k2j1FudfAgQNZvHgxhw4d4rXXXmPZsmUkJSWRlJRktlrDMIxTIOZvGCJSF1gDVAWOi8hwvICCPYEJwNl4uS7+LCL7VPUl4GVgvK8vA+jlv22cMl26dCEtLY1KlSpRxCYMwzAMYthh6Im83eDlswByHFSDOeGguiZkQvwl/9qjIrIHuET9SXPDMAwjWAINPqgFOKhEZAuwD+/t4wX1Muvl1dbPgg/mDjr4/fffM3r0aKZOnRqDTxM+JcFtEStc1gamL1Jc1ueyNigmLqmCNsJwUHEi+GAdYB3QpbB283NJbdmyRVu3bn0KnoHYUBLcFrHCZW2qpi9SXNbnsjbVkhV8MF/0RPDBXcA8PHvtKTN48GDat2/Ppk2boinPMAyjxBFUh1EFWCki74jIPLyV3UNFpA2AiFQSkVdFZJefurUH3tqMU+bbb79FRDhy5AgJCQm89NJLUfsQhmEYJYmg1mFUxUu7OgS4CW+BXgVgjR83qhbwC+BH4Fzgr6q6uCg3Wrp0KWlpafTp04cNG4rU5xiGYRgEl6JVgVfw4k31VNX/8899A1RQLwfGeSFhQx6Nt07DMAzjZOI+JKWqtwPbgW7AROAaABHpgBfVNiH/qwsmd/BBwzAMI3oEndP7KF6ncQHwGV62vVtUdZ1frzG5AhPm0VahwQfNVhseLutzWRuYvkhxWZ/L2qAE2mr9MvHLq4aUNSZXYMKCNrPVRobL+lzWpmr6IsVlfS5rUy1BtloRqSYiZf3DW4AVqro/mvdo0qQJ5557Lp9//rm5pAzDMCIgKJdUFWAlXtjy1n7e7qN4w1IAiEgy0BkoLSI/AfeqHzrkVJg2bRqVK1fm5ptvNpeUYRhGBAT1hrEbL/jgdcB+oJmqVgX6AYhIabwItc3w0rWmAauKcqMuXbpQo0aNKEg2DMMo2cS9w8iVvvUPeHkv/gcnpWLtAGxS1c2qehSYDfQtrG1zSRmGYcSOoF1SfwbOAFrjDVNNVNVXRaQ/XkjzW/z6vwU6qupdebRlwQejhMv6XNYGpi9SXNbnsjYoQS4p4FlgNVDJP/4abxjqN8CUkPq/BZ4prF1zSUWGy/pc1qZq+iLFZX0ua1MtWSlatwF7VDUdSBeRFUCiX94wpF4C3mK/U2bw4MG8/fbbpKenRyzWMAyjJBNo8EHgV8AfRGSdiHyBNxG+EdgJdBWRL/zyP+DNeZwyFnzQMAwjOgQdfLCXr+EsvIV71YGv/Dq/4USa1kp4yZROGQs+aBiGER2CckllBx9UvDeK1sDlwA7gmKoeVdX5qtoMuAg4EG+dhmEYxskE7ZI6gjfU1AJvmOo6VX3Hr9MQeAc4D7hPVf+RT1vmkooSLutzWRuYvkhxWZ/L2qBkuaT6A0/hDUedh7fyu2quuvWBj4CzCmvXXFKR4bI+l7Wpmr5IcVmfy9pUS1AsKWAQ3sI9VdVNeB1Gi9AK6qVq/Ry4JAB9hmEYhk/QHcb/8JxSiMhZQHNgs4gkiEgFv7w68EsgtSg3GDBgABdffDGpqanmkjIMw4iAoNdh/BWYJiKf4Q1L3a+qe0SkOzBBRNQvH6+qnxXUUH7MmjUremoNwzBKMIF0GKraOOSwRx7nlwBt4ybIMAzDKJSgh6QMwzCM04RAbLWxQkQOUMS5jjhRC9gTtIgCcFmfy9rA9EWKy/pc1gbR0ddIVWsXVinoOYxok6rheIkDQkTWmL6i4bI2MH2R4rI+l7VBfPXZkJRhGIYRFtZhGIZhGGFR3DqMF4MWUAimr+i4rA1MX6S4rM9lbRBHfcVq0tswDMOIHcXtDcMwDMOIEdZhGIZhGGFRLDoMEeklIqkisklERjmgp6GILBeRjSLyuYgM88triMgSEfna/1k9YJ2lReRTEVnoH58jIh/6+v4lImUD1FZNRN4QkS/953ixK89PRO7x/103iMgsESkf9LMTkZdFZJeIbAgpy/N5icck//dlvYi0C0DbOP/fdr2IzBORaiHnRvvaUkWkZyy15acv5NwIEVERqeUfx/XZFaRPRO72n9HnIvJkSHnsnl84IW1d3oDSwDdAE6AssA5oFbCmekA7f78KXhbBVsCTwCi/fBTwRMA67wVmAgv94znA9f7+ZOCOALW9Atzi75cFqrnw/IAGeFGVK4Q8s4FBPzugC9AO2BBSlufzwktW9i5enLZOwIcBaOsBlPH3nwjR1sr/HS4HnOP/bpeOtz6/vCHwb+BboFYQz66A59cNeB8o5x/Xicfzi9t/6Bg+zIuBf4ccjwZGB60rl8a38VLSpgL1/LJ6eAsNg9KUACwFLgMW+r8Ae0J+iU96rnHWVtX/UpZc5YE/P7/D2ArUwFv4uhAvF33gzw5onOtLJc/nBbwADMirXry05TrXD5jh75/0++t/YV8c72fnl70BJOLn7wnq2eXzbzsH+HUe9WL6/IrDkFT2L3A22/wyJxCRxsAFwId4SaB2APg/6wSnjKeBkcBx/7gm8KOqHvOPg3yOTYDdwFR/yGyKiFTCgeenqt/h5Zr/H15K4Z+Atbjz7ELJ73m59jszGO+vdnBEm4hcBXynqutynXJCH9AMuMQfBv1ARC7yy2Oqrzh0GJJHmRNeYRGpDMwFhqvq/qD1ZCMifYBdqro2tDiPqkE9xzJ4r+DPq+oFQDrekErg+PMAffFe9+sDlYDeeVR14v9gPjjzby0iDwDHgBnZRXlUi6s2EakIPAD8Ja/TeZQF8ezKANXxhsXuA+aIiBBjfcWhw9iGN9aYTQKwPSAtOYjIGXidxQxVfdMv3iki9fzz9YBdAcn7JXCVeLnVZ+MNSz0NVBOR7PhiQT7HbcA2Vf3QP34DrwNx4fn9GtiiqrtVNRN4E/gF7jy7UPJ7Xk78zojI74A+wI3qj2paoxsAAAN+SURBVJ84ou1cvD8I1vm/IwnAJyJS1xF9+Dqys5V+hDdSUCvW+opDh/Ex0NR3qZQFrgfmBynI7+lfAjaq6t9DTs0Hfufv/w5vbiPuqOpoVU1QLy/J9cAyVb0RWI6XZz1ofd8DW0WkuV/0K+AL3Hh+/wM6iUhF/985W5sTzy4X+T2v+cDNvuOnE/BT9tBVvBCRXsD9wFWqeijk1HzgehEpJyLnAE2Bj+KpTVU/U9U6qtrY/x3Zhmdi+R4Hnp3PW3h/6CEizfCMIXuI9fOL9WRNPDY858JXeI6ABxzQ0xnvNXA9kOJvl+PNEywFvvZ/1nBAa1dOuKSa+P+5NgGv4zswAtKVBKzxn+FbeK/fTjw/4GHgS2AD8BqeIyXQZwfMwptTycT7ghuS3/PCG7b4h//78hlwYQDaNuGNtWf/fkwOqf+Ary0V6B3Es8t1Po0Tk95xfXYFPL+ywHT//+AnwGXxeH4WGsQwDMMIi+IwJGUYhmHEAeswDMMwjLCwDsMwDMMIC+swDMMwjLCwDsMwDMMIizKFVzGMko2IZOFZKLO5WlXTApJjGIFhtlrDKAQROaiqleN4vzJ6Ii6VYTiDDUkZRoSISD0RWSEiKX6OjEv88l4i8omIrBORpX5ZDRF5y8+lsFpE2vrlD4nIiyLyHvCqeLlKxonIx37d2wL8iIYB2JCUYYRDBRFJ8fe3qGq/XOdvwAtn/qiIlAYqikht4J9AF1XdIiI1/LoPA5+q6tUichnwKt6qdoD2QGdVzRCRW/HCTlwkIuWA/4rIe6q6JZYf1DAKwjoMwyicDFVNKuD8x8DLfsDJt1Q1RUS6Aiuyv+BV9Qe/bmfgWr9smYjUFJEz/XPzVTXD3+8BtBWR7PhUZ+LFBbIOwwgM6zAMI0JUdYWIdAGuAF4TkXHAj+QdVrqg8NPpuer9//buEAeBGIqi6P1sBMcC2ABj8HgSPAKJZx+wBjwWg0OwjUESDKKINoEQEpoMBHOPrOqol7aT/xYppd1XNyt14BuG1FFE9Mn9ImvylOIhcABGZWIoT1dSe2Ba1hrgnN53peyAeTm1EBGDUiIl/Y0nDKm7BlhGxA24ALOUUlveIbYR0SN3UYyBFblJ8ARceYwff7Uh13Ieyxj1Fpj88iOkT/ytVpJUxSspSVIVA0OSVMXAkCRVMTAkSVUMDElSFQNDklTFwJAkVbkDjWgnPWOC1ecAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 显示重要特征\n",
    "plot_importance(xgb_test)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#from graphviz import Digraph\n",
    "#import pydot\n",
    "#显示树图\n",
    "#xgb_test.fit(X_test, y_test)\n",
    "\n",
    "#plot_tree(xgb_test)\n",
    "#plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'base_score': 0.5,\n",
       " 'booster': 'gbtree',\n",
       " 'colsample_bylevel': 0.7,\n",
       " 'colsample_bytree': 0.9,\n",
       " 'gamma': 0,\n",
       " 'learning_rate': 0.1,\n",
       " 'max_delta_step': 0,\n",
       " 'max_depth': 5,\n",
       " 'min_child_weight': 1,\n",
       " 'missing': None,\n",
       " 'n_estimators': 250,\n",
       " 'nthread': 1,\n",
       " 'objective': 'multi:softprob',\n",
       " 'reg_alpha': 0,\n",
       " 'reg_lambda': 1,\n",
       " 'scale_pos_weight': 1,\n",
       " 'seed': 3,\n",
       " 'silent': 1,\n",
       " 'subsample': 0.8}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb_test.get_xgb_params()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#保存模型\n",
    "import pickle\n",
    "pickle.dump(xgb_test, open(\"xgb_model.pkl\", 'wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of train is: 1.1075368226258033\n"
     ]
    }
   ],
   "source": [
    "#加载模型\n",
    "import pickle\n",
    "\n",
    "xgb = pickle.load(open(\"xgb_model.pkl\", 'rb'))\n",
    "\n",
    "train_predprob = xgb.predict_proba(X_train)\n",
    "logloss = log_loss(y_train, train_predprob)\n",
    "\n",
    "#Print model report:\n",
    "print ('logloss of train is:', logloss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "加载训练好的模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb = pickle.load(open(\"xgb_model.pkl\", 'rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import pandas as pd \n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>virtual</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2950</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3758</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3300</td>\n",
       "      <td>1650.000000</td>\n",
       "      <td>1100.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "      <td>4900</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 227 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.0         1   2950      1475.000000     1475.000000        0.0   \n",
       "1        1.0         2   2850      1425.000000      950.000000       -1.0   \n",
       "2        1.0         1   3758      1879.000000     1879.000000        0.0   \n",
       "3        1.0         2   3300      1650.000000     1100.000000       -1.0   \n",
       "4        2.0         2   4900      1633.333333     1633.333333        0.0   \n",
       "\n",
       "   room_num  Year  Month  Day  ...   virtual  walk  walls  war  washer  water  \\\n",
       "0       2.0  2016      6   11  ...         0     0      0    0       0      0   \n",
       "1       3.0  2016      6   24  ...         0     0      0    1       0      0   \n",
       "2       2.0  2016      6    3  ...         0     0      0    0       0      0   \n",
       "3       3.0  2016      6   11  ...         0     0      0    0       0      0   \n",
       "4       4.0  2016      4   12  ...         0     0      0    1       0      0   \n",
       "\n",
       "   wheelchair  wifi  windows  work  \n",
       "0           0     0        0     0  \n",
       "1           0     0        0     0  \n",
       "2           0     0        0     0  \n",
       "3           1     0        0     0  \n",
       "4           0     0        0     0  \n",
       "\n",
       "[5 rows x 227 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#读取测试数据\n",
    "test = pd.read_csv(\"RentListingInquries_FE_test.csv\")\n",
    "test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_id = test.index\n",
    "\n",
    "X_test = np.array(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "#测试并提交结果\n",
    "y_test_pred = xgb_test.predict_proba(X_test)\n",
    "\n",
    "out_df1 = pd.DataFrame(y_test_pred)\n",
    "out_df1.columns = [\"high\", \"medium\", \"low\"]\n",
    "\n",
    "out_df = pd.concat([test,out_df1], axis = 1)\n",
    "out_df.to_csv(\"xgb_Rent.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
